-- Logs begin at Sun 2022-08-21 00:10:36 UTC, end at Sun 2022-08-21 01:46:59 UTC. -- Aug 21 01:16:02.115467 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Aug 21 01:16:02.127523 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Aug 21 01:16:02.130143 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Aug 21 01:16:02 2022] *** Aug 21 01:16:02.130295 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 21 01:16:02.130405 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 21 01:16:02.130497 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: nodename: dv-cinder-master-3-iscsi Aug 21 01:16:02.130601 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: machine: x86_64 Aug 21 01:16:02.130693 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: clock source: unix Aug 21 01:16:02.130787 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: pcre jit disabled Aug 21 01:16:02.130875 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: detected number of CPU cores: 4 Aug 21 01:16:02.130963 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: current working directory: / Aug 21 01:16:02.131057 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: detected binary path: /usr/bin/uwsgi-core Aug 21 01:16:02.131171 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: your processes number limit is 47680 Aug 21 01:16:02.131171 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: your memory page size is 4096 bytes Aug 21 01:16:02.131171 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: detected max file descriptor number: 2048 Aug 21 01:16:02.131171 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: lock engine: pthread robust mutexes Aug 21 01:16:02.131466 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: thunder lock: enabled Aug 21 01:16:02.131532 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Aug 21 01:16:02.131605 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 21 01:16:02.170931 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: Python main interpreter initialized at 0x55abbc8a19c0 Aug 21 01:16:02.170931 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: python threads support enabled Aug 21 01:16:02.170931 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: your server socket listen backlog is limited to 100 connections Aug 21 01:16:02.170931 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: your mercy for graceful operations on workers is 90 seconds Aug 21 01:16:02.171377 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: mapped 403077 bytes (393 KB) for 2 cores Aug 21 01:16:02.171377 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: *** Operational MODE: preforking *** Aug 21 01:16:02.171644 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: *** uWSGI is running in multiple interpreter mode *** Aug 21 01:16:02.171644 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: spawned uWSGI master process (pid: 106139) Aug 21 01:16:02.171917 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Aug 21 01:16:02.172824 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: spawned uWSGI worker 1 (pid: 106140, cores: 1) Aug 21 01:16:02.173703 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: spawned uWSGI worker 2 (pid: 106141, cores: 1) Aug 21 01:16:02.173703 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 21 01:16:02.837934 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 21 01:16:02.865819 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 21 01:16:03.955155 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 21 01:16:03.955155 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: from cryptography.utils import int_from_bytes Aug 21 01:16:03.955860 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 21 01:16:03.955860 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: from cryptography.utils import int_from_bytes Aug 21 01:16:04.045737 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 21 01:16:04.045737 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: from cryptography.utils import int_from_bytes Aug 21 01:16:04.046611 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 21 01:16:04.046611 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: from cryptography.utils import int_from_bytes Aug 21 01:16:06.496775 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 21 01:16:06.497370 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 21 01:16:06.497574 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 21 01:16:06.498138 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 21 01:16:06.498399 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 21 01:16:06.498792 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.499120 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.499220 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 21 01:16:06.499576 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 21 01:16:06.499674 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.499768 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.499943 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 21 01:16:06.500066 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.500339 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 21 01:16:06.500436 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.500603 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 21 01:16:06.500697 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.500994 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.501092 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.501323 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.501416 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.501581 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.501675 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.501893 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.501987 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.502218 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.502324 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.502540 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.502649 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.502851 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.502957 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.503162 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.503259 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.503482 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.503579 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.503742 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.503834 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.504058 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.504185 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.504408 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.504507 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.504747 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.504895 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.505153 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.505533 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.505668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.505788 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.505906 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.506014 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.506134 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.506300 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.506446 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.506613 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.506766 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.506907 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.507227 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.507360 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.507534 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.507671 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.507858 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.507998 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.508188 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.508327 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.508530 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.508659 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.508867 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.509001 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.509179 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.509313 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.509501 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.509639 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.509813 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.509948 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.510140 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.510276 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.510453 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.510588 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.510847 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.510981 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.511164 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.511301 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.511477 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.511611 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.511873 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.512008 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.512156 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.512301 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.512440 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.512622 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.512751 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.512968 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.513105 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.513254 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.513856 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.514097 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.514223 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.514430 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.514545 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.514765 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.514982 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.515108 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.515226 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.515341 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.515459 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.515644 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.515770 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.515967 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.516121 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.516318 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.516442 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.516629 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.516749 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.516878 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.516998 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.517111 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.517246 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.517424 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.517570 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.517738 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.517874 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.518039 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.518173 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.518348 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.518488 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.518656 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.518790 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.518965 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.519093 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.519260 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.519394 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.519524 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.519654 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.519821 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.519949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.520164 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.520306 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.520498 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.520624 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.520812 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.520955 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.521129 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.521265 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.521433 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.521559 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.521748 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.521924 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.522062 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.522189 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.47 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.522350 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.522479 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.47 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.522661 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.522802 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.522983 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.523142 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.523286 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.523419 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.523602 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.523739 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.523923 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.524094 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.524310 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.524444 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.524583 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.524708 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.47 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.524837 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.524982 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.47 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.525176 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.525313 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.525451 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.525583 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.525742 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.525871 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.526045 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.526180 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.526314 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.526447 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.526572 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.526871 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.527212 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.527351 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.527497 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.527623 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.527755 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.527755 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.527997 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.528175 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.528304 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.528428 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.528552 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.528675 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.528798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.529516 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.529651 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.529786 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.529918 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.530042 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.530165 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.530350 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.530487 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.530653 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.530782 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.530957 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.531086 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.531262 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.531391 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.531564 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.531706 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.531873 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.532049 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.532236 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.532363 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.532527 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.532657 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.532836 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.532973 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.533147 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.533275 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.533454 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.533583 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.533762 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.533892 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.534072 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.534201 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.534377 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.534511 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.534697 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.534833 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.535000 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.535126 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.535332 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.535470 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.535606 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.535744 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.535898 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.536035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.536210 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.536344 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.536529 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.536667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.536831 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.536961 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.537137 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.537272 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.537480 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.537616 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.537790 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.537923 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.538092 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.538219 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.538396 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.538533 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.538700 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.538836 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.539004 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.539140 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.539307 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.539516 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.539649 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.539784 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.539920 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.540054 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.540214 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.540341 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.540504 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.540640 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.540823 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.540959 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.541130 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.541267 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.541434 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.541556 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.541736 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.541872 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.542073 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.542205 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.542330 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.542453 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.542639 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.542775 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.542942 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.543077 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.543263 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.543391 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.543585 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.543728 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.543895 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.544023 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.544250 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.544392 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.544589 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.544726 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.544915 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:16:06.545043 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.545239 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.545458 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.545587 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.545723 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.545859 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.545992 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.546147 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.546283 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.546466 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.546595 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.546783 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.546918 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.547113 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.547241 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.547432 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.547567 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.547749 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.547878 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.548069 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.548228 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.548384 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.548520 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.548705 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.548837 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.549019 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.549141 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.549339 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.549439 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.549651 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.549750 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.549978 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.550071 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.550300 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.550388 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.550617 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.550702 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.550960 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.551058 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.551278 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.551367 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.551603 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.551745 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.551934 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.552073 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.552271 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.552407 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.552576 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.552713 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.552907 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.553050 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.553220 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.553348 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.553540 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.553670 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.553857 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.553994 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.554170 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.554307 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.554482 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.554610 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.554819 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.554956 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.555133 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.555269 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.555459 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.555601 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.555776 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.555910 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.556186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.556330 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.556601 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.556737 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.556921 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.557240 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.557369 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.557555 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.557684 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.557898 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.558032 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.558274 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.558401 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.558751 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.558888 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.559069 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.559212 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.559402 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.559529 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.559724 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.559853 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.560049 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.560251 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.560470 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.560601 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.560795 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.561039 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.561179 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.561418 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.561572 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.561731 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.562042 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.562246 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.562390 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.562654 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.562789 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.562981 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.563559 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.563683 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.563884 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.564139 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.564286 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.564473 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.564602 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.564792 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.564929 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.565116 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.565253 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.565428 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.565577 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.565795 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.565924 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.566056 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.566180 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.566346 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.566475 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.566656 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.566790 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.566981 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.567108 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.567289 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.567416 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.567597 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.567733 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.567913 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.568047 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.568249 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.568386 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.568563 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.568693 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.568883 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.569011 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.569199 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.569341 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.569519 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.569647 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.569837 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.569965 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.570157 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.570299 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.570486 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.570614 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.570805 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.570934 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.571129 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.571266 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.571467 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.571594 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.571874 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.572002 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.572218 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.572390 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.572527 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.572651 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.572825 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.572954 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.573134 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.573264 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.573443 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.573568 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.573771 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.573901 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.574083 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.574253 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.574382 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.574690 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.574821 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.575006 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.575218 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.575346 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.575643 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.576334 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.576553 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.576684 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.576819 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.576979 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.577108 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.577363 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.577489 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.577681 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.577953 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.578139 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.578277 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.578488 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.578629 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.578911 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.579060 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.579207 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.579493 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.579671 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.579889 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.580129 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.580329 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.580530 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.580745 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.580963 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.581171 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.581391 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.581583 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.581799 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.582010 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.582153 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.582285 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.582443 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.582569 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.582569 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.582816 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.582948 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.583173 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.583300 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.583300 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.583544 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.583668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.583792 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.583915 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.584023 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.584279 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.584435 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.584652 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.584792 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.584964 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.585102 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.585287 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.585417 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.585632 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.585762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.585971 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.586112 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.586292 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.586432 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.586623 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.586773 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.586943 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.587092 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.587262 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.587454 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.587611 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.587852 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.587992 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.588205 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.588363 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.588525 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.588671 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.588851 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.588992 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.589166 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.589332 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.589473 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.589699 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.589848 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.590066 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.590206 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.590401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.590541 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.590725 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.590867 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.591034 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.591175 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.591353 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.592051 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.592238 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.592426 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.592608 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.592797 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.592937 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.593181 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.593330 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.593505 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.593646 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.593819 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.593963 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.594134 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.594284 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.594434 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.594620 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.594768 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.595010 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.595156 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.595341 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.595480 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.595660 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.595810 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.595971 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.596290 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.596436 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.596577 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.596729 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.596951 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.597107 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.597259 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.597407 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.597622 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.597766 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.597903 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.598053 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.598201 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.598352 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.598489 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.598642 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.598792 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.599013 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.599164 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.599381 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.599535 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.599674 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.599809 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.599963 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.600175 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.600329 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.600465 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.600618 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.600824 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.600984 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.601182 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.601329 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.601473 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.601608 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.601818 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.601976 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.602141 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.602285 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.602451 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.602628 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.602863 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.603021 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.603192 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.603369 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.603524 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.603733 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.603886 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.604093 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.604279 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.604423 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.604566 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.604750 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.604924 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.605063 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.605215 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.605351 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.605543 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.605688 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.605904 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.606061 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.606245 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.606383 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.606559 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.606705 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.606886 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.607138 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.607280 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.607420 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.607566 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.607741 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.607881 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.608129 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.608272 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.608442 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.608582 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.608765 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.608920 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.609239 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.609381 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.609624 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.609771 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.609986 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.610126 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.610310 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.610453 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.610633 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.610783 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.610957 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.611099 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.611292 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.611443 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.611626 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.611770 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.611976 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.612157 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.612317 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.612498 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.612690 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.612862 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.613021 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.613204 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.613342 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.613519 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.613662 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.613827 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.613977 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.614119 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.614291 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.614443 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.614727 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.614910 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.615049 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.615253 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.615383 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.615574 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.615704 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.615899 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.616037 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.616246 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.616388 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.616553 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.616691 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.616869 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.617008 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.617180 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.617314 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.617511 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.617620 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.617825 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.617931 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.618132 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.618229 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.618817 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.618977 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.619165 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.619319 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.619419 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.619501 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.619501 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.619647 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.619818 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.619975 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.620197 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.620358 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.620515 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.620643 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.620797 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.620948 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.621109 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.621242 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.621418 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.621556 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.621729 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.621944 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.622072 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.622196 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.622344 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.622474 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.622734 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.622872 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.623004 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.623130 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.623316 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.623453 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.623629 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.623759 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.623973 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.624133 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.624277 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.624397 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.624561 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.624692 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.624865 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.625003 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.625180 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.625310 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.625489 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.625627 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.625807 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.625938 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.626120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.626258 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.626427 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.626556 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.626733 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.626864 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.627048 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.627187 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.627363 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.627476 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.627668 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.627764 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.627974 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.628189 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.628307 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.628457 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.628620 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.628749 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.628924 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.629075 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.629231 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.629361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.629538 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.629665 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.629849 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.629977 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.630156 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.630284 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.630459 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.630589 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.630770 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.630900 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.631073 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.631203 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.631378 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.631505 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.631684 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.631813 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.631988 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.632177 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.632308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.632433 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.632613 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.632753 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.632925 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.633052 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.633211 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.633337 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.633521 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.633652 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.633826 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.633956 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.634138 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.634280 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.634448 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.634569 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.634752 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.634881 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.635057 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.635201 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.635366 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.635502 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.635673 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.635795 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.635974 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.636133 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.636307 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.636421 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.636616 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.636732 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.636919 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.637035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.637222 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.637338 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.637566 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.637699 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.637817 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.637932 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.638122 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.638772 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.638969 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.639101 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.639294 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.639423 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.639614 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.639745 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.639919 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.640057 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.640285 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.640414 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.640540 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.640667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.640840 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.640979 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.641145 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.641274 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.641455 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.641592 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.641759 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.641898 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.642064 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.642194 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.642367 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.642496 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.642671 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.642810 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.642980 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.643122 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.643285 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.643407 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.643592 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.643730 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.643896 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.644045 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.644227 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.644358 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.644531 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.644669 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.644851 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.644987 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.645175 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.645309 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.645503 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.645631 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.645811 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.645942 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.646125 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.646253 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.646438 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.646575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.646747 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.646875 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.647081 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.647216 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.647395 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.647525 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.647701 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.647823 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.648016 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.648155 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.648348 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.648453 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.648660 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.648746 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.648967 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.649118 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.649273 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.649416 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.649570 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.649709 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.649954 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.650112 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.650256 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.650361 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.650492 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.650631 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.650813 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.651080 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.651239 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.651379 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.651517 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.651657 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.651822 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.651968 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.652151 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.652293 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.652432 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.652570 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.652734 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.652876 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.653016 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.653168 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.653308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.653470 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.653619 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.653757 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.653906 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.654619 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.654797 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.654949 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.655102 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.655254 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.655398 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.655535 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.655676 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.655818 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.655974 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.656233 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.656403 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.656562 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.656706 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.656856 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.656998 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.657139 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.657279 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.657435 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.657578 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.657728 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.657958 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.658106 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.658247 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.658388 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.658529 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.658680 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.658832 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.658974 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.659113 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.659264 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.659415 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.659558 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.659698 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.659838 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.659977 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.660154 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.660320 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.660576 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.660726 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.660894 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.661042 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.661190 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.661350 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.661602 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.661752 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.661917 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.662064 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.662228 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.662383 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.662526 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.662667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.662835 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.662983 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.663199 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.663349 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.663501 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.663670 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.663812 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.663952 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.664133 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.664282 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.664423 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.664574 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.664716 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.664864 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.665039 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.665188 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.665377 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.665535 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.665676 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.665817 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.666051 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.666201 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.666440 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.666622 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.666775 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.666918 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.667086 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.667236 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.667386 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.667540 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.667681 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.667821 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.668015 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.668176 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.668358 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.668455 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.668683 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.668784 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.668989 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.669089 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.669303 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.669392 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.669640 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.669728 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.669967 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.670065 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.670282 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.670381 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.670589 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.670676 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.670904 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.671004 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.671207 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.671300 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.671522 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.671608 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.671838 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.671926 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.672169 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.672260 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.672589 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.672687 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.672836 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.672943 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.673142 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.673227 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.673469 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.673557 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.673786 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.673894 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.674113 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.674204 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.674439 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.674526 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.674744 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.674832 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.675055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.675144 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.675361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.675457 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.675666 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.675751 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.675973 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.676067 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.676305 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.676425 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.676619 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.676714 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.676930 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.677026 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.677240 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.677342 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.677546 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.677633 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.677849 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.677948 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.678174 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.678270 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.678479 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.678568 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.678802 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.678896 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.679115 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.679203 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.679438 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.679524 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.679759 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.679846 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.680064 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.680192 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.680408 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.680506 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.680716 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.680802 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.681023 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.681111 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.681327 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.681421 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.681632 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.681720 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.681945 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.682031 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.682285 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.682382 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.682599 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.682698 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.682909 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.682998 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.683241 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.683329 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.683552 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.683649 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.683883 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.683989 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.684222 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.684326 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.684535 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.684632 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.684845 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.684933 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.685176 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.685262 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.685529 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.685625 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.685832 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.685925 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.686165 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.686251 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.686494 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.686639 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.686797 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.686922 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.687106 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.687222 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.687403 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.687530 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.687708 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.687839 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.688006 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.688204 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.688591 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.688718 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.688893 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.689039 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.689176 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.689302 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.689509 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.689611 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.689821 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.689983 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.690119 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.690261 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.690423 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.690568 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.690781 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.691045 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.691126 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.691356 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.691499 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.691687 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.691845 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.691987 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.692214 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.692381 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.692526 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.692683 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.692939 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.693029 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.693251 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.693370 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.693570 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.693667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.693889 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.694082 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.694201 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.694521 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.694666 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.694832 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.695131 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.695229 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.695445 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.695712 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.695807 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.696314 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.696406 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.696654 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.696754 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.696971 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.697285 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.697380 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.697599 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.697874 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.697966 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.698211 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.698436 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.698547 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.698820 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.699024 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.699120 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.699429 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.699528 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.699760 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.699881 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.700081 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.700244 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.700430 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.700543 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.701310 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.701585 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.701707 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.701934 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.702087 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.702229 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.702321 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.702544 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.702642 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.702905 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.703057 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.703222 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.703323 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.703528 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.703624 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.703842 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.703930 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.704181 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.704295 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.704496 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.704584 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.704809 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.704896 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.705124 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.705229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.705436 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.705523 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.705746 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.705841 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.706059 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.706147 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.706367 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.706454 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.706679 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.706767 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.706992 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.707283 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.707372 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.707610 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.707697 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.707944 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.708040 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.708283 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.708373 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.708597 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.708684 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.708913 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.709002 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.709246 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.709443 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.709585 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.709765 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.709900 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.710106 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.710202 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.710436 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.710530 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.710766 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.710862 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.711087 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.711192 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.711423 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.711509 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.711730 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.711825 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.712037 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.712155 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.712370 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.712457 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.712684 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.712771 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.712995 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.713221 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.713315 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.713575 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.713672 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.713898 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.713985 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.714528 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.714642 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.714720 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.714803 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.714883 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.717311 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.717311 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.717493 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.717576 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.717679 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.717764 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.717844 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.717979 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.718060 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.718150 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.718240 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.718320 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.718407 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.718488 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.718575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.718670 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.718752 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.718832 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.718919 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.719008 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.719089 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.719178 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.719266 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.719353 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.719434 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.719819 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.719913 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.720009 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.720092 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.720188 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.720269 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.720361 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.720512 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.720666 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.720820 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.720986 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.721133 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.721295 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.721440 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.721618 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.721768 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.721909 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.722077 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.722216 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.722383 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.722531 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.722689 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.722837 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.723001 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.723156 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.723328 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.723465 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.723600 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.723780 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.723927 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.724088 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.724252 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.724421 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.724543 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.724726 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.724849 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.725052 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.725165 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.725362 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.725471 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.725719 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.725865 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.726024 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.726174 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.726342 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.726436 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.726658 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.726745 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.726971 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.727059 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.727275 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.727369 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.727625 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.727725 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.727946 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.728042 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.728280 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.728532 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.728620 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.728873 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.728969 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.729197 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.729293 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.729615 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.729712 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.729855 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.729943 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.730172 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.730403 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.730491 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.730738 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.730835 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.731060 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.731147 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.731388 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.731474 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.731729 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.731829 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.732063 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.732224 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.732464 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.732614 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.732769 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.732906 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.733084 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.733224 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.733399 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.733546 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.733701 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.733854 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.734009 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.734099 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.734321 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.734418 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.734631 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.734718 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.734939 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.735026 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.735253 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.735361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.735560 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.735648 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.735872 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.735959 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.736207 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.736311 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.736520 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.736608 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.736831 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.736916 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.737143 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.737231 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.737453 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.737540 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.737767 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.737862 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.738083 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.738170 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.738398 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.738482 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.738710 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.738796 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.739029 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.739116 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.739349 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.739425 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.739662 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.739738 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.739967 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.740052 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.740302 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.740428 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.740615 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.740746 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.740933 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.741068 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.741264 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.741393 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.741581 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.741707 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.741887 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.742031 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.742307 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.742384 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.742623 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.742709 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.742943 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.743019 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.743265 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.743343 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.743579 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.743656 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.743902 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.743978 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.744257 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.744338 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.744640 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.744732 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.744889 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.745061 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.745202 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.745362 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.745512 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.745601 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.746247 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.746353 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.746435 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.746525 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.746619 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.746757 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.746877 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.746962 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.747041 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.747120 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.747338 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.747424 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.747678 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.747769 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.747990 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.748088 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.748326 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.748413 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.748630 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.748728 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.748933 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.749022 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.749237 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.749329 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.749541 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.749628 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.749848 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.749934 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.750150 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.750235 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.750455 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.750542 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.750761 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.750849 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.751061 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.751155 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.751366 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.751461 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.751669 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.751755 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.751983 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.752186 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.752321 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.752585 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.752745 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.752878 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.753010 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.753135 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.753278 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.753388 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.753470 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.753541 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.753755 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.753851 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.754053 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.754164 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.754354 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.754491 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.754687 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.754803 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.755021 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.755119 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.755339 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.755428 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.755662 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.755749 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.755977 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.756066 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.756325 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.756433 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.756636 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.756731 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.756958 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.757047 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.757282 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.757384 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.757594 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.757681 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.757904 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.757991 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.758213 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.758298 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.758536 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.758621 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.758852 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.759079 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.759174 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.759279 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.759459 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.759610 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.759770 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.759932 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.760082 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.760366 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.760514 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.760691 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.760820 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.761007 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.761117 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.761348 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.761498 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.761652 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.761797 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.761971 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.762140 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.762288 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.762393 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.762582 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.762682 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.762891 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.762986 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.763208 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.763801 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.764037 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.764161 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.764374 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.764474 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.764689 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.764786 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.765001 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.765089 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.765317 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.765403 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.765634 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.765723 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.765944 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.766039 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.766260 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.766347 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.766575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.766663 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.766904 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.766991 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.767229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.767315 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.767538 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.767625 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.767865 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.767951 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.768211 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.768303 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.768535 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.768622 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.768850 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.768993 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.769134 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=106140) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 21 01:16:06.769258 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.769516 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.769831 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.770147 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.770453 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:16:06.770699 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=106141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 21 01:16:06.927508 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO dbcounter [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Registered counter for database nova_api Aug 21 01:16:06.928286 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO dbcounter [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Registered counter for database nova_api Aug 21 01:16:06.951426 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG oslo_db.sqlalchemy.engines [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106141) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:16:06.952738 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_db.sqlalchemy.engines [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106140) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:16:06.960045 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writer thread running {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:16:06.960402 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG dbcounter [-] [106141] Writer thread running {{(pid=106141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:16:07.016296 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.context [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),6547be85-e237-4d35-aaa7-703fe51c4e0b(cell1) {{(pid=106141) load_cells /opt/stack/nova/nova/context.py:464}} Aug 21 01:16:07.016595 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.context [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),6547be85-e237-4d35-aaa7-703fe51c4e0b(cell1) {{(pid=106140) load_cells /opt/stack/nova/nova/context.py:464}} Aug 21 01:16:07.020160 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:16:07.023131 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG oslo_concurrency.lockutils [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106141) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:16:07.023270 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:16:07.023361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG oslo_concurrency.lockutils [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=106141) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:16:07.023458 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:16:07.023555 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG oslo_concurrency.lockutils [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=106141) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:16:07.023640 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:16:07.023733 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG oslo_concurrency.lockutils [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106141) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:16:07.023818 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:16:07.023902 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG oslo_concurrency.lockutils [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106141) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:16:07.025406 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:16:07.025562 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG oslo_concurrency.lockutils [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=106141) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:16:07.027348 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO dbcounter [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Registered counter for database nova_cell0 Aug 21 01:16:07.027504 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO dbcounter [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Registered counter for database nova_cell0 Aug 21 01:16:07.033358 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO dbcounter [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Registered counter for database nova_cell1 Aug 21 01:16:07.033631 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO dbcounter [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Registered counter for database nova_cell1 Aug 21 01:16:07.040740 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG oslo_db.sqlalchemy.engines [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106141) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:16:07.042542 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_db.sqlalchemy.engines [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106140) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:16:07.043962 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG oslo_db.sqlalchemy.engines [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106141) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:16:07.045973 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_db.sqlalchemy.engines [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106140) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:16:07.051051 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG dbcounter [-] [106141] Writer thread running {{(pid=106141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:16:07.051315 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writer thread running {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:16:07.052559 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG dbcounter [-] [106141] Writer thread running {{(pid=106141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:16:07.192362 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writer thread running {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:16:07.203464 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO dbcounter [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Registered counter for database nova_cell0 Aug 21 01:16:07.210941 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO dbcounter [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Registered counter for database nova_cell0 Aug 21 01:16:07.213131 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_db.sqlalchemy.engines [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106140) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:16:07.218116 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writer thread running {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:16:07.219872 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG oslo_db.sqlalchemy.engines [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106141) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:16:07.224121 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG dbcounter [-] [106141] Writer thread running {{(pid=106141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:16:07.622559 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: WARNING keystonemiddleware.auth_token [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 21 01:16:07.630383 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: WARNING keystonemiddleware.auth_token [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 21 01:16:07.654367 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: WARNING keystonemiddleware.auth_token [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 21 01:16:07.662080 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: WARNING keystonemiddleware.auth_token [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 21 01:16:07.970335 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: WARNING keystonemiddleware.auth_token [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 21 01:16:07.976324 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: WARNING keystonemiddleware.auth_token [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 21 01:16:07.981544 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55abbc8a19c0 pid: 106140 (default app) Aug 21 01:16:07.984209 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:16:07.985398 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] 10.222.0.47 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001775 Aug 21 01:16:07.985814 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 788 bytes} [Sun Aug 21 01:16:07 2022] GET /compute/v2.1/ => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:16:08.009510 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: WARNING keystonemiddleware.auth_token [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 21 01:16:08.016043 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: WARNING keystonemiddleware.auth_token [None req-ba80a66d-0952-4709-9d61-b737c6cec8c2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 21 01:16:08.021316 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55abbc8a19c0 pid: 106141 (default app) Aug 21 01:17:32.134847 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:17:32.135748 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001403 Aug 21 01:17:32.136368 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 2/2] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:17:32 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:17:32.149089 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG dbcounter [-] [106141] Writing DB stats nova_api:SELECT=3 {{(pid=106141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:17:32.149994 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG dbcounter [-] [106141] Writing DB stats nova_cell1:SELECT=3 {{(pid=106141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:17:32.151451 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG dbcounter [-] [106141] Writing DB stats nova_cell0:SELECT=3 {{(pid=106141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:17:32.152344 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG dbcounter [-] [106141] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=106141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:17:33.025580 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-d8a60f59-926f-4849-9d03-006a7c5f2d96 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:17:33.058769 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-d8a60f59-926f-4849-9d03-006a7c5f2d96 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.915474 Aug 21 01:17:33.060129 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 1/3] 10.222.0.47 () {60 vars in 1292 bytes} [Sun Aug 21 01:17:32 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 917 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:17:35.419794 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:17:35.420506 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-22e8564b-84c4-4171-8721-6f3516bfe41a None None] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001766 Aug 21 01:17:35.423928 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 3/4] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:17:35 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:17:35.916062 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-8a35391d-dca5-4fce-b3d0-3fa3b111c21c admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "disk": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 256, "rxtx_factor": 1.0, "id": "c1", "os-flavor-access:is_public": true, "vcpus": 1, "name": "cirros256"}} {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:35.937585 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 21 01:17:35.937585 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: warnings.warn(msg) Aug 21 01:17:36.034215 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-8a35391d-dca5-4fce-b3d0-3fa3b111c21c admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.606744 Aug 21 01:17:36.035292 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 2/5] 10.222.0.47 () {64 vars in 1283 bytes} [Sun Aug 21 01:17:35 2022] POST /compute/v2.1/flavors => generated 407 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:17:36.056273 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writing DB stats nova_api:SELECT=3 {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:17:36.057587 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writing DB stats nova_cell0:SELECT=3 {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:17:36.059315 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writing DB stats nova_cell1:SELECT=3 {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:17:36.060769 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:17:36.269780 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-df08cbd4-a7b7-4156-b48a-a458289571ca admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:36.277089 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 21 01:17:36.277089 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: warnings.warn(msg) Aug 21 01:17:36.406462 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-df08cbd4-a7b7-4156-b48a-a458289571ca admin admin] 10.222.0.47 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.358985 Aug 21 01:17:36.407050 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 4/6] 10.222.0.47 () {64 vars in 1336 bytes} [Sun Aug 21 01:17:36 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:17:38.798510 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-8a35391d-dca5-4fce-b3d0-3fa3b111c21c admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:17:38.799510 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-8a35391d-dca5-4fce-b3d0-3fa3b111c21c admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001469 Aug 21 01:17:38.800291 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 3/7] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:17:38 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:17:39.472915 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-fa2b535e-ba8c-4b72-9488-755d4e3ae2e7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "rxtx_factor": 1.0, "name": "ds512M", "vcpus": 1, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d1", "swap": 0, "disk": 5}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:39.493399 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 21 01:17:39.493399 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: warnings.warn(msg) Aug 21 01:17:39.585831 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-fa2b535e-ba8c-4b72-9488-755d4e3ae2e7 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.778629 Aug 21 01:17:39.586867 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 5/8] 10.222.0.47 () {64 vars in 1283 bytes} [Sun Aug 21 01:17:38 2022] POST /compute/v2.1/flavors => generated 404 bytes in 781 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 21 01:17:39.607469 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-948ecc85-5796-4e04-8f9c-90cfb920ccb3 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:39.611749 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 21 01:17:39.611749 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: warnings.warn(msg) Aug 21 01:17:39.731883 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-948ecc85-5796-4e04-8f9c-90cfb920ccb3 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.131297 Aug 21 01:17:39.732943 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 4/9] 10.222.0.47 () {64 vars in 1336 bytes} [Sun Aug 21 01:17:39 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:17:42.138202 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-fa2b535e-ba8c-4b72-9488-755d4e3ae2e7 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:17:42.139920 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-fa2b535e-ba8c-4b72-9488-755d4e3ae2e7 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002497 Aug 21 01:17:42.140656 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 6/10] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:17:42 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:17:42.556090 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-7beec7fc-6318-4b37-8ac5-67e239213a6a admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds1G", "disk": 10, "rxtx_factor": 1.0, "vcpus": 1, "ram": 1024, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "id": "d2"}} {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:42.627316 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-7beec7fc-6318-4b37-8ac5-67e239213a6a admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.477501 Aug 21 01:17:42.627868 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 5/11] 10.222.0.47 () {64 vars in 1283 bytes} [Sun Aug 21 01:17:42 2022] POST /compute/v2.1/flavors => generated 404 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 21 01:17:42.639788 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-188ece7c-5c5b-4e89-bef8-3cc445b17ec8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:42.729707 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-188ece7c-5c5b-4e89-bef8-3cc445b17ec8 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.093544 Aug 21 01:17:42.730753 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 7/12] 10.222.0.47 () {64 vars in 1336 bytes} [Sun Aug 21 01:17:42 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:17:45.126496 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-7beec7fc-6318-4b37-8ac5-67e239213a6a admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:17:45.127442 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-7beec7fc-6318-4b37-8ac5-67e239213a6a admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Aug 21 01:17:45.127850 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 6/13] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:17:45 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:17:45.717878 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-db69029d-4b9e-46c6-aa8b-b2023bdaa3a1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "ram": 2048, "disk": 10, "vcpus": 2, "name": "ds2G", "id": "d3", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "swap": 0}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:45.804865 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-db69029d-4b9e-46c6-aa8b-b2023bdaa3a1 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.669207 Aug 21 01:17:45.805830 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 8/14] 10.222.0.47 () {64 vars in 1283 bytes} [Sun Aug 21 01:17:45 2022] POST /compute/v2.1/flavors => generated 404 bytes in 671 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:17:45.819799 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-98b47ede-ce38-4534-8951-f0ef568c5e1e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:45.905043 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-98b47ede-ce38-4534-8951-f0ef568c5e1e admin admin] 10.222.0.47 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.089526 Aug 21 01:17:45.905867 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 7/15] 10.222.0.47 () {64 vars in 1336 bytes} [Sun Aug 21 01:17:45 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:17:48.440256 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-db69029d-4b9e-46c6-aa8b-b2023bdaa3a1 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:17:48.441786 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-db69029d-4b9e-46c6-aa8b-b2023bdaa3a1 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002022 Aug 21 01:17:48.442359 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 9/16] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:17:48 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:17:48.915681 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-db6fe104-d926-4d3a-8156-d530544b0baa admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 4096, "rxtx_factor": 1.0, "name": "ds4G", "os-flavor-access:is_public": true, "vcpus": 4, "id": "d4", "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0}} {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:49.057458 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-db6fe104-d926-4d3a-8156-d530544b0baa admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.606836 Aug 21 01:17:49.058389 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 8/17] 10.222.0.47 () {64 vars in 1283 bytes} [Sun Aug 21 01:17:48 2022] POST /compute/v2.1/flavors => generated 404 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 21 01:17:49.072916 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-6acb7e1b-60c1-4386-b4f6-6212ec3bc546 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:49.151577 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-6acb7e1b-60c1-4386-b4f6-6212ec3bc546 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.083311 Aug 21 01:17:49.152193 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 10/18] 10.222.0.47 () {64 vars in 1336 bytes} [Sun Aug 21 01:17:49 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:17:51.676459 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-db6fe104-d926-4d3a-8156-d530544b0baa admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:17:51.677552 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-db6fe104-d926-4d3a-8156-d530544b0baa admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001777 Aug 21 01:17:51.678503 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 9/19] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:17:51 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:17:52.187249 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-2ce2e02d-164d-4bfe-954a-9b3b8ac6aa5a admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:17:52.213666 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-2ce2e02d-164d-4bfe-954a-9b3b8ac6aa5a admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.528591 Aug 21 01:17:52.214218 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 11/20] 10.222.0.47 () {60 vars in 1292 bytes} [Sun Aug 21 01:17:51 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:17:54.783563 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-db6fe104-d926-4d3a-8156-d530544b0baa admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:17:54.784938 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-db6fe104-d926-4d3a-8156-d530544b0baa admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001976 Aug 21 01:17:54.785532 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 10/21] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:17:54 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:17:55.322556 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-511bd79c-df3c-4416-b76b-a859fcdc07bb admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "name": "m1.tiny", "ram": 512, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "disk": 1, "id": "1", "swap": 0}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:55.400478 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-511bd79c-df3c-4416-b76b-a859fcdc07bb admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.608351 Aug 21 01:17:55.400959 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 12/22] 10.222.0.47 () {64 vars in 1283 bytes} [Sun Aug 21 01:17:54 2022] POST /compute/v2.1/flavors => generated 402 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:17:55.414082 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-0aa4f43f-07f4-4505-ac32-2097fe619163 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:55.494665 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-0aa4f43f-07f4-4505-ac32-2097fe619163 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.085032 Aug 21 01:17:55.495245 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 11/23] 10.222.0.47 () {64 vars in 1333 bytes} [Sun Aug 21 01:17:55 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:17:57.908053 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-511bd79c-df3c-4416-b76b-a859fcdc07bb admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:17:57.908708 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-511bd79c-df3c-4416-b76b-a859fcdc07bb admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001445 Aug 21 01:17:57.909051 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 13/24] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:17:57 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:17:58.415382 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1f1090f6-4a4f-4b1e-be68-b1c700b28724 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "id": "2", "ram": 2048, "os-flavor-access:is_public": true, "swap": 0, "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "name": "m1.small"}} {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:58.513467 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1f1090f6-4a4f-4b1e-be68-b1c700b28724 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.597265 Aug 21 01:17:58.513941 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 12/25] 10.222.0.47 () {64 vars in 1283 bytes} [Sun Aug 21 01:17:57 2022] POST /compute/v2.1/flavors => generated 405 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:17:58.526415 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-56129b0d-5edc-467e-85da-6d6abac9a484 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:17:58.637007 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-56129b0d-5edc-467e-85da-6d6abac9a484 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.114330 Aug 21 01:17:58.637642 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 14/26] 10.222.0.47 () {64 vars in 1333 bytes} [Sun Aug 21 01:17:58 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:18:01.231293 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1f1090f6-4a4f-4b1e-be68-b1c700b28724 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:01.241835 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1f1090f6-4a4f-4b1e-be68-b1c700b28724 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003733 Aug 21 01:18:01.241835 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 13/27] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:18:01 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:18:01.751430 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-f587e952-cd1d-4c9c-848d-f9616b62b927 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "name": "m1.medium", "disk": 40, "vcpus": 2, "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "swap": 0, "id": "3"}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:18:02.080149 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-f587e952-cd1d-4c9c-848d-f9616b62b927 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.836631 Aug 21 01:18:02.087896 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 15/28] 10.222.0.47 () {64 vars in 1283 bytes} [Sun Aug 21 01:18:01 2022] POST /compute/v2.1/flavors => generated 406 bytes in 838 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:18:02.093885 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-b751a0b4-e54c-4d79-ac14-28a9696898ee admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:18:02.211087 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-b751a0b4-e54c-4d79-ac14-28a9696898ee admin admin] 10.222.0.47 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.121645 Aug 21 01:18:02.212077 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 14/29] 10.222.0.47 () {64 vars in 1333 bytes} [Sun Aug 21 01:18:02 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:18:04.651540 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-f587e952-cd1d-4c9c-848d-f9616b62b927 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:04.653369 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-f587e952-cd1d-4c9c-848d-f9616b62b927 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002359 Aug 21 01:18:04.656615 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 16/30] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:18:04 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:18:05.169697 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-140144f8-543b-47b5-bfee-b105f1c9ba12 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.large", "id": "4", "os-flavor-access:is_public": true, "vcpus": 4, "ram": 8192, "swap": 0, "disk": 80, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:18:05.182464 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG dbcounter [-] [106141] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=106141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:18:05.293696 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-140144f8-543b-47b5-bfee-b105f1c9ba12 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.633322 Aug 21 01:18:05.294261 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 15/31] 10.222.0.47 () {64 vars in 1283 bytes} [Sun Aug 21 01:18:04 2022] POST /compute/v2.1/flavors => generated 405 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:18:05.307320 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-102ca375-110a-4891-ac5f-ba3e8cd5f7a4 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:18:05.390649 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-102ca375-110a-4891-ac5f-ba3e8cd5f7a4 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.086814 Aug 21 01:18:05.391407 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 17/32] 10.222.0.47 () {64 vars in 1333 bytes} [Sun Aug 21 01:18:05 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:18:07.884351 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-140144f8-543b-47b5-bfee-b105f1c9ba12 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:07.887255 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-140144f8-543b-47b5-bfee-b105f1c9ba12 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003573 Aug 21 01:18:07.892191 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 16/33] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:18:07 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:18:08.411310 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-6d5d3e9f-65f7-4dcb-bee8-779c0025a7b5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 8, "ram": 16384, "disk": 160, "os-flavor-access:is_public": true, "swap": 0, "id": "5", "name": "m1.xlarge", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:18:08.491000 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-6d5d3e9f-65f7-4dcb-bee8-779c0025a7b5 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.594556 Aug 21 01:18:08.491000 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 18/34] 10.222.0.47 () {64 vars in 1283 bytes} [Sun Aug 21 01:18:07 2022] POST /compute/v2.1/flavors => generated 408 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:18:08.506324 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-ebeb4a81-448f-4e4a-acda-9f8cf4a56e94 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:18:08.624066 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-ebeb4a81-448f-4e4a-acda-9f8cf4a56e94 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.122688 Aug 21 01:18:08.640641 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 17/35] 10.222.0.47 () {64 vars in 1333 bytes} [Sun Aug 21 01:18:08 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:18:27.416675 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-6d5d3e9f-65f7-4dcb-bee8-779c0025a7b5 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:27.417720 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-6d5d3e9f-65f7-4dcb-bee8-779c0025a7b5 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001502 Aug 21 01:18:27.418123 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 19/36] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:18:27 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:18:27.815139 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-cbf5d0af-2cf0-4d47-866d-78d624467a23 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:27.835886 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-cbf5d0af-2cf0-4d47-866d-78d624467a23 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.411577 Aug 21 01:18:27.836453 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 18/37] 10.222.0.47 () {60 vars in 1292 bytes} [Sun Aug 21 01:18:27 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:18:28.902867 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-6d5d3e9f-65f7-4dcb-bee8-779c0025a7b5 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:28.903866 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-6d5d3e9f-65f7-4dcb-bee8-779c0025a7b5 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001409 Aug 21 01:18:28.904268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 20/38] 10.222.0.47 () {56 vars in 1074 bytes} [Sun Aug 21 01:18:28 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:18:29.443474 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-491decf7-f121-4016-9d72-71256821b8d3 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:29.449350 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.wsgi [None req-491decf7-f121-4016-9d72-71256821b8d3 admin admin] HTTP exception thrown: Flavor 42 could not be found. Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-491decf7-f121-4016-9d72-71256821b8d3 admin admin] Request method failure captured: Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Accept: application/json Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Accept-Encoding: gzip, deflate Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Connection: keep-alive Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Content-Length: 0 Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Host: 10.222.0.47 Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Openstack-Api-Version: compute 2.90 Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Openstack-System-Scope: None Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: User-Agent: python-novaclient Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Auth-Token: gAAAAABjAYfkimD5LBJDKz-bo_8KToaG_95jmRAmfwUJGAj9kKmGctfz0Whb3SRDi_UXCL92LszCnC9pkJfmB9LlsRw-QVWGz_UnSojhj6tGgo8yMzMJZ6ZNjSb9w1DheqK22o-ZZDJulvZ4l9Ro7jUkgMvn0WJFUxMEemAF1zB_j7BwJneGoO4 Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Domain-Id: None Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Domain-Name: None Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Identity-Status: Confirmed Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Is-Admin-Project: True Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Openstack-Nova-Api-Version: 2.90 Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Project-Domain-Id: default Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Project-Domain-Name: Default Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Project-Id: ada79d28652f4a3aa0a174133f24108f Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Project-Name: admin Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Role: member,reader,admin Aug 21 01:18:29.452575 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Roles: member,reader,admin Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Tenant: admin Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Tenant-Id: ada79d28652f4a3aa0a174133f24108f Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Tenant-Name: admin Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User: admin Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User-Domain-Id: default Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User-Domain-Name: Default Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User-Id: b3234aad965e427281c9352b840a0824 Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User-Name: admin Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: method: > Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: exception: Flavor 42 could not be found. Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: action_args: {'id': '42'} Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:18:29.454308 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:18:29.455371 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:29.456762 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-491decf7-f121-4016-9d72-71256821b8d3 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=106141) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:18:29.457827 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-491decf7-f121-4016-9d72-71256821b8d3 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.399403 Aug 21 01:18:29.457827 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 19/39] 10.222.0.47 () {60 vars in 1190 bytes} [Sun Aug 21 01:18:29 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 400 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 21 01:18:29.463612 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-c7e8c749-2f44-4377-9fd5-e088dfdfec07 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:29.467820 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:18:29.486373 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-c7e8c749-2f44-4377-9fd5-e088dfdfec07 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.026237 Aug 21 01:18:29.486925 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 21/40] 10.222.0.47 () {60 vars in 1210 bytes} [Sun Aug 21 01:18:29 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:18:29.497246 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-559660c4-04cb-4aed-bc76-1b9a41e48b30 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:29.521635 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-559660c4-04cb-4aed-bc76-1b9a41e48b30 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.027839 Aug 21 01:18:29.522184 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 20/41] 10.222.0.47 () {60 vars in 1210 bytes} [Sun Aug 21 01:18:29 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:18:29.531638 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-193e4cff-fdc5-47ab-8776-be52615ce8f3 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:29.537411 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.wsgi [None req-193e4cff-fdc5-47ab-8776-be52615ce8f3 admin admin] HTTP exception thrown: Flavor 42 could not be found. Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-193e4cff-fdc5-47ab-8776-be52615ce8f3 admin admin] Request method failure captured: Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Accept: application/json Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Accept-Encoding: gzip, deflate Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Connection: keep-alive Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Content-Length: 0 Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Host: 10.222.0.47 Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Openstack-Api-Version: compute 2.90 Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Openstack-System-Scope: None Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: User-Agent: python-novaclient Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Auth-Token: gAAAAABjAYfkimD5LBJDKz-bo_8KToaG_95jmRAmfwUJGAj9kKmGctfz0Whb3SRDi_UXCL92LszCnC9pkJfmB9LlsRw-QVWGz_UnSojhj6tGgo8yMzMJZ6ZNjSb9w1DheqK22o-ZZDJulvZ4l9Ro7jUkgMvn0WJFUxMEemAF1zB_j7BwJneGoO4 Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Domain-Id: None Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Domain-Name: None Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Identity-Status: Confirmed Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Is-Admin-Project: True Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Openstack-Nova-Api-Version: 2.90 Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Project-Domain-Id: default Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Project-Domain-Name: Default Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Project-Id: ada79d28652f4a3aa0a174133f24108f Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Project-Name: admin Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Role: member,reader,admin Aug 21 01:18:29.540667 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Roles: member,reader,admin Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Tenant: admin Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Tenant-Id: ada79d28652f4a3aa0a174133f24108f Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Tenant-Name: admin Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User: admin Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User-Domain-Id: default Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User-Domain-Name: Default Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User-Id: b3234aad965e427281c9352b840a0824 Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User-Name: admin Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: method: > Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: exception: Flavor 42 could not be found. Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: action_args: {'id': '42'} Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:18:29.542281 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:18:29.543120 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:29.544275 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-193e4cff-fdc5-47ab-8776-be52615ce8f3 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=106140) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:18:29.545298 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-193e4cff-fdc5-47ab-8776-be52615ce8f3 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.014111 Aug 21 01:18:29.545298 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 22/42] 10.222.0.47 () {60 vars in 1190 bytes} [Sun Aug 21 01:18:29 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 21 01:18:30.532399 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-559660c4-04cb-4aed-bc76-1b9a41e48b30 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:30.533439 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-559660c4-04cb-4aed-bc76-1b9a41e48b30 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001526 Aug 21 01:18:30.533840 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 21/43] 10.222.0.47 () {56 vars in 1074 bytes} [Sun Aug 21 01:18:30 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:18:31.077721 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-b12c96da-bf8c-475d-9d3a-aa5c1d50a40d admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:31.083545 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.wsgi [None req-b12c96da-bf8c-475d-9d3a-aa5c1d50a40d admin admin] HTTP exception thrown: Flavor 84 could not be found. Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-b12c96da-bf8c-475d-9d3a-aa5c1d50a40d admin admin] Request method failure captured: Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Accept: application/json Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Accept-Encoding: gzip, deflate Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Connection: keep-alive Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Content-Length: 0 Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Host: 10.222.0.47 Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Openstack-Api-Version: compute 2.90 Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Openstack-System-Scope: None Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: User-Agent: python-novaclient Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Auth-Token: gAAAAABjAYfmCdP4P014_4tau8yOjYP1V4S6U8vPPFM29iw8hANspV0EWO_XgDYzQr2wg1VN3QtWMTvgllPzk3BOvtvJPbgTJ2yrcUAELC-0PTFNwr0PlkwnNQJrUFkBP1BQbOBJmGGLKf09RtAUI8uYj0G_AkrJWIBrDkE2fPfJ2njRcek2490 Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Domain-Id: None Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Domain-Name: None Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Identity-Status: Confirmed Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Is-Admin-Project: True Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Openstack-Nova-Api-Version: 2.90 Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Project-Domain-Id: default Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Project-Domain-Name: Default Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Project-Id: ada79d28652f4a3aa0a174133f24108f Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Project-Name: admin Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Role: member,reader,admin Aug 21 01:18:31.085035 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Roles: member,reader,admin Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Tenant: admin Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Tenant-Id: ada79d28652f4a3aa0a174133f24108f Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Tenant-Name: admin Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User: admin Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User-Domain-Id: default Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User-Domain-Name: Default Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User-Id: b3234aad965e427281c9352b840a0824 Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User-Name: admin Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: method: > Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: exception: Flavor 84 could not be found. Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: action_args: {'id': '84'} Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:18:31.086534 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:18:31.087401 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:31.088798 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-b12c96da-bf8c-475d-9d3a-aa5c1d50a40d admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=106140) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:18:31.090058 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-b12c96da-bf8c-475d-9d3a-aa5c1d50a40d admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.413764 Aug 21 01:18:31.090058 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 23/44] 10.222.0.47 () {60 vars in 1190 bytes} [Sun Aug 21 01:18:30 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 415 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 21 01:18:31.096982 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-d6d6a118-e80e-4104-9503-4a5f5e08a927 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:31.111714 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-d6d6a118-e80e-4104-9503-4a5f5e08a927 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018364 Aug 21 01:18:31.112775 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 22/45] 10.222.0.47 () {60 vars in 1210 bytes} [Sun Aug 21 01:18:31 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:18:31.122204 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-de0bdda8-80a0-4299-a31c-15445fd0d895 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:31.138265 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-de0bdda8-80a0-4299-a31c-15445fd0d895 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.019549 Aug 21 01:18:31.138895 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 24/46] 10.222.0.47 () {60 vars in 1210 bytes} [Sun Aug 21 01:18:31 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:18:31.149194 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-181b77ea-ed84-4c8a-ba00-a6cd72b60c8e admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:31.154618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.wsgi [None req-181b77ea-ed84-4c8a-ba00-a6cd72b60c8e admin admin] HTTP exception thrown: Flavor 84 could not be found. Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-181b77ea-ed84-4c8a-ba00-a6cd72b60c8e admin admin] Request method failure captured: Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Accept: application/json Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Accept-Encoding: gzip, deflate Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Connection: keep-alive Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Content-Length: 0 Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Host: 10.222.0.47 Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Openstack-Api-Version: compute 2.90 Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Openstack-System-Scope: None Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: User-Agent: python-novaclient Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Auth-Token: gAAAAABjAYfmCdP4P014_4tau8yOjYP1V4S6U8vPPFM29iw8hANspV0EWO_XgDYzQr2wg1VN3QtWMTvgllPzk3BOvtvJPbgTJ2yrcUAELC-0PTFNwr0PlkwnNQJrUFkBP1BQbOBJmGGLKf09RtAUI8uYj0G_AkrJWIBrDkE2fPfJ2njRcek2490 Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Domain-Id: None Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Domain-Name: None Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Identity-Status: Confirmed Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Is-Admin-Project: True Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Openstack-Nova-Api-Version: 2.90 Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Project-Domain-Id: default Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Project-Domain-Name: Default Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Project-Id: ada79d28652f4a3aa0a174133f24108f Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Project-Name: admin Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Role: member,reader,admin Aug 21 01:18:31.156186 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Roles: member,reader,admin Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Tenant: admin Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Tenant-Id: ada79d28652f4a3aa0a174133f24108f Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Tenant-Name: admin Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User: admin Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User-Domain-Id: default Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User-Domain-Name: Default Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User-Id: b3234aad965e427281c9352b840a0824 Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User-Name: admin Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: method: > Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: exception: Flavor 84 could not be found. Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: action_args: {'id': '84'} Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:18:31.160949 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:18:31.163055 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:31.166361 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-181b77ea-ed84-4c8a-ba00-a6cd72b60c8e admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=106141) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:18:31.167441 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-181b77ea-ed84-4c8a-ba00-a6cd72b60c8e admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.012416 Aug 21 01:18:31.167441 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 23/47] 10.222.0.47 () {60 vars in 1190 bytes} [Sun Aug 21 01:18:31 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 21 01:18:32.144995 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-de0bdda8-80a0-4299-a31c-15445fd0d895 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:32.146000 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-de0bdda8-80a0-4299-a31c-15445fd0d895 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001446 Aug 21 01:18:32.146416 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 25/48] 10.222.0.47 () {56 vars in 1074 bytes} [Sun Aug 21 01:18:32 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:18:32.697287 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-90c890f9-d466-48da-b543-260c6f68fcae admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:32.703455 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.wsgi [None req-90c890f9-d466-48da-b543-260c6f68fcae admin admin] HTTP exception thrown: Flavor 451 could not be found. Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-90c890f9-d466-48da-b543-260c6f68fcae admin admin] Request method failure captured: Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Accept: application/json Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Accept-Encoding: gzip, deflate Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Connection: keep-alive Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Content-Length: 0 Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Host: 10.222.0.47 Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Openstack-Api-Version: compute 2.90 Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Openstack-System-Scope: None Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: User-Agent: python-novaclient Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Auth-Token: gAAAAABjAYfo6HmFDWMYQ0UqAiuib0xKcAlmUioBsQCLtO4XnfPaV5PEqemOc6atsrdb8IoRCsuiuJgXE-8I_2ByD6uQbODsLHKfjSoN9aum77gKO6i4qnJNWt6U07jqAtImgv40tpiqoT6RkXl04JeHsGw5QhmS3gr8KO3zXAzHo8imfE8aO90 Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Domain-Id: None Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Domain-Name: None Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Identity-Status: Confirmed Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Is-Admin-Project: True Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Openstack-Nova-Api-Version: 2.90 Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Project-Domain-Id: default Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Project-Domain-Name: Default Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Project-Id: ada79d28652f4a3aa0a174133f24108f Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Project-Name: admin Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Role: member,reader,admin Aug 21 01:18:32.705012 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Roles: member,reader,admin Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Tenant: admin Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Tenant-Id: ada79d28652f4a3aa0a174133f24108f Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-Tenant-Name: admin Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User: admin Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User-Domain-Id: default Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User-Domain-Name: Default Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User-Id: b3234aad965e427281c9352b840a0824 Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: X-User-Name: admin Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: method: > Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: exception: Flavor 451 could not be found. Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: action_args: {'id': '451'} Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:18:32.706758 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:18:32.707618 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: ERROR nova.api.openstack.wsgi Aug 21 01:18:32.708869 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-90c890f9-d466-48da-b543-260c6f68fcae admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=106141) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:18:32.710188 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-90c890f9-d466-48da-b543-260c6f68fcae admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.414149 Aug 21 01:18:32.710188 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 24/49] 10.222.0.47 () {60 vars in 1193 bytes} [Sun Aug 21 01:18:32 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 415 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 21 01:18:32.716259 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-ec96ed74-e457-4599-a008-75fa2e74f38b admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:32.739671 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-ec96ed74-e457-4599-a008-75fa2e74f38b admin admin] 10.222.0.47 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.026658 Aug 21 01:18:32.740592 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 26/50] 10.222.0.47 () {60 vars in 1210 bytes} [Sun Aug 21 01:18:32 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:18:32.752808 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:32.767927 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018620 Aug 21 01:18:32.768496 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 25/51] 10.222.0.47 () {60 vars in 1210 bytes} [Sun Aug 21 01:18:32 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:18:32.778091 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-9e788d3b-c089-4f26-9114-453339b79016 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:32.784151 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.wsgi [None req-9e788d3b-c089-4f26-9114-453339b79016 admin admin] HTTP exception thrown: Flavor 451 could not be found. Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-9e788d3b-c089-4f26-9114-453339b79016 admin admin] Request method failure captured: Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Accept: application/json Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Accept-Encoding: gzip, deflate Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Connection: keep-alive Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Content-Length: 0 Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Host: 10.222.0.47 Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Openstack-Api-Version: compute 2.90 Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Openstack-System-Scope: None Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: User-Agent: python-novaclient Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Auth-Token: gAAAAABjAYfo6HmFDWMYQ0UqAiuib0xKcAlmUioBsQCLtO4XnfPaV5PEqemOc6atsrdb8IoRCsuiuJgXE-8I_2ByD6uQbODsLHKfjSoN9aum77gKO6i4qnJNWt6U07jqAtImgv40tpiqoT6RkXl04JeHsGw5QhmS3gr8KO3zXAzHo8imfE8aO90 Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Domain-Id: None Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Domain-Name: None Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Identity-Status: Confirmed Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Is-Admin-Project: True Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Openstack-Nova-Api-Version: 2.90 Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Project-Domain-Id: default Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Project-Domain-Name: Default Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Project-Id: ada79d28652f4a3aa0a174133f24108f Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Project-Name: admin Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Role: member,reader,admin Aug 21 01:18:32.785655 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Roles: member,reader,admin Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ada79d28652f4a3aa0a174133f24108f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Tenant: admin Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Tenant-Id: ada79d28652f4a3aa0a174133f24108f Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-Tenant-Name: admin Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User: admin Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User-Domain-Id: default Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User-Domain-Name: Default Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User-Id: b3234aad965e427281c9352b840a0824 Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: X-User-Name: admin Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: method: > Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: exception: Flavor 451 could not be found. Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: action_args: {'id': '451'} Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:18:32.787229 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:18:32.788268 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: ERROR nova.api.openstack.wsgi Aug 21 01:18:32.789668 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-9e788d3b-c089-4f26-9114-453339b79016 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=106140) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:18:32.790906 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-9e788d3b-c089-4f26-9114-453339b79016 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.012498 Aug 21 01:18:32.790906 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 27/52] 10.222.0.47 () {60 vars in 1193 bytes} [Sun Aug 21 01:18:32 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 14 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 21 01:18:33.765245 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:33.766259 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001504 Aug 21 01:18:33.766660 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 26/53] 10.222.0.47 () {56 vars in 1074 bytes} [Sun Aug 21 01:18:33 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:18:34.295747 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-d72d90db-b640-48a0-ae27-9fbfe574acff admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:18:34.373590 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-d72d90db-b640-48a0-ae27-9fbfe574acff admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.468446 Aug 21 01:18:34.374149 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 28/54] 10.222.0.47 () {64 vars in 1235 bytes} [Sun Aug 21 01:18:33 2022] POST /compute/v2.1/flavors => generated 404 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:18:35.357874 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:35.358868 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001425 Aug 21 01:18:35.359254 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 27/55] 10.222.0.47 () {56 vars in 1074 bytes} [Sun Aug 21 01:18:35 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:18:35.900990 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-8ef3ad49-bdd6-458e-a084-be6236b16f40 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:18:35.972874 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-8ef3ad49-bdd6-458e-a084-be6236b16f40 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.474278 Aug 21 01:18:35.973426 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 29/56] 10.222.0.47 () {64 vars in 1235 bytes} [Sun Aug 21 01:18:35 2022] POST /compute/v2.1/flavors => generated 405 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:18:36.969756 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:36.970820 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001544 Aug 21 01:18:36.971221 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 28/57] 10.222.0.47 () {56 vars in 1074 bytes} [Sun Aug 21 01:18:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:18:37.520688 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-f3408d03-3ea8-4e55-b56c-d7992f29ca7e admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:18:37.601584 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-f3408d03-3ea8-4e55-b56c-d7992f29ca7e admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.487834 Aug 21 01:18:37.602127 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 30/58] 10.222.0.47 () {64 vars in 1235 bytes} [Sun Aug 21 01:18:37 2022] POST /compute/v2.1/flavors => generated 407 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:18:38.572747 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:38.573739 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001455 Aug 21 01:18:38.574124 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 29/59] 10.222.0.47 () {56 vars in 1074 bytes} [Sun Aug 21 01:18:38 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:18:39.103101 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-63ca915f-4efb-4c97-9a53-6095f9053128 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:18:39.167184 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-63ca915f-4efb-4c97-9a53-6095f9053128 admin admin] 10.222.0.47 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.453471 Aug 21 01:18:39.167792 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 31/60] 10.222.0.47 () {64 vars in 1235 bytes} [Sun Aug 21 01:18:38 2022] POST /compute/v2.1/flavors => generated 409 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:18:40.144913 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:40.145902 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002004 Aug 21 01:18:40.146305 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 30/61] 10.222.0.47 () {56 vars in 1074 bytes} [Sun Aug 21 01:18:40 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:18:40.673904 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-da27b64a-1059-4724-aa25-5afe6d2515a2 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:18:40.695759 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-da27b64a-1059-4724-aa25-5afe6d2515a2 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.413841 Aug 21 01:18:40.696327 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 32/62] 10.222.0.47 () {60 vars in 1202 bytes} [Sun Aug 21 01:18:40 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:19:35.125750 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:19:35.126760 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1a4708e4-399e-457c-97dd-c30bc1003580 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001452 Aug 21 01:19:35.127180 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 31/63] 10.222.0.47 () {54 vars in 931 bytes} [Sun Aug 21 01:19:35 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:19:35.137651 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:19:35.537831 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-34153702-cca5-4007-9afc-4067663b3704 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:19:35.558654 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-34153702-cca5-4007-9afc-4067663b3704 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.425301 Aug 21 01:19:35.559116 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 33/64] 10.222.0.47 () {60 vars in 1292 bytes} [Sun Aug 21 01:19:35 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:19:35.580899 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG dbcounter [-] [106141] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=22,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=106141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:19:35.584654 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-c7edead0-cc53-4f1c-a2c9-d5a80e3ed72b admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:19:35.592811 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-c7edead0-cc53-4f1c-a2c9-d5a80e3ed72b admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012714 Aug 21 01:19:35.597550 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 32/65] 10.222.0.47 () {60 vars in 1283 bytes} [Sun Aug 21 01:19:35 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:19:35.605133 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-e81b6834-b18b-4ded-bba7-cb56282d0943 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:19:35.613471 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-e81b6834-b18b-4ded-bba7-cb56282d0943 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011883 Aug 21 01:19:35.614018 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 34/66] 10.222.0.47 () {60 vars in 1286 bytes} [Sun Aug 21 01:19:35 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:19:35.626279 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1c98c93c-5e7b-4bc9-91f0-998b15ac9989 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:19:35.634431 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1c98c93c-5e7b-4bc9-91f0-998b15ac9989 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011736 Aug 21 01:19:35.640785 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 33/67] 10.222.0.47 () {60 vars in 1286 bytes} [Sun Aug 21 01:19:35 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:19:35.647918 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-9d8d22ed-6288-4f0e-8a92-f3acac8ef519 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:19:35.656509 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-9d8d22ed-6288-4f0e-8a92-f3acac8ef519 admin admin] 10.222.0.47 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012135 Aug 21 01:19:35.657092 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 35/68] 10.222.0.47 () {60 vars in 1283 bytes} [Sun Aug 21 01:19:35 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:23:02.234832 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1c98c93c-5e7b-4bc9-91f0-998b15ac9989 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:23:02.235833 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1c98c93c-5e7b-4bc9-91f0-998b15ac9989 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001438 Aug 21 01:23:02.236254 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 34/69] 10.222.0.47 () {56 vars in 1074 bytes} [Sun Aug 21 01:23:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:23:02.378600 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writing DB stats nova_api:SELECT=6 {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:23:02.524826 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-8e45def2-dda1-4d3e-a963-bb12d327a227 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:23:02.527363 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 21 01:23:02.527363 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: warnings.warn(msg) Aug 21 01:23:02.528508 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-8e45def2-dda1-4d3e-a963-bb12d327a227 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:23:02.529020 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-8e45def2-dda1-4d3e-a963-bb12d327a227 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:23:02.529473 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-8e45def2-dda1-4d3e-a963-bb12d327a227 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:23:02.530192 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-8e45def2-dda1-4d3e-a963-bb12d327a227 admin admin] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:23:02.530641 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-8e45def2-dda1-4d3e-a963-bb12d327a227 admin admin] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:23:02.531061 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-8e45def2-dda1-4d3e-a963-bb12d327a227 admin admin] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:23:02.567057 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-8e45def2-dda1-4d3e-a963-bb12d327a227 admin admin] 10.222.0.47 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.189771 Aug 21 01:23:02.567630 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 36/70] 10.222.0.47 () {60 vars in 1193 bytes} [Sun Aug 21 01:23:02 2022] GET /compute/v2.1/os-services => generated 791 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:23:03.402041 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1c98c93c-5e7b-4bc9-91f0-998b15ac9989 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:23:03.403023 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1c98c93c-5e7b-4bc9-91f0-998b15ac9989 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001400 Aug 21 01:23:03.403411 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 35/71] 10.222.0.47 () {56 vars in 1074 bytes} [Sun Aug 21 01:23:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:23:03.597458 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-12ad4752-c385-460d-9467-340c1b6c0b78 admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:23:03.599849 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 21 01:23:03.599849 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: warnings.warn(msg) Aug 21 01:23:03.601040 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-12ad4752-c385-460d-9467-340c1b6c0b78 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:23:03.601538 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-12ad4752-c385-460d-9467-340c1b6c0b78 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:23:03.602040 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-12ad4752-c385-460d-9467-340c1b6c0b78 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:23:03.602750 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-12ad4752-c385-460d-9467-340c1b6c0b78 admin admin] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:23:03.603193 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-12ad4752-c385-460d-9467-340c1b6c0b78 admin admin] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:23:03.603612 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-12ad4752-c385-460d-9467-340c1b6c0b78 admin admin] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:23:03.623531 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-12ad4752-c385-460d-9467-340c1b6c0b78 admin admin] 10.222.0.47 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.082906 Aug 21 01:23:03.624028 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 37/72] 10.222.0.47 () {60 vars in 1193 bytes} [Sun Aug 21 01:23:03 2022] GET /compute/v2.1/os-services => generated 791 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:23:14.857015 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1c98c93c-5e7b-4bc9-91f0-998b15ac9989 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:23:14.858025 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1c98c93c-5e7b-4bc9-91f0-998b15ac9989 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001419 Aug 21 01:23:14.858416 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 36/73] 10.222.0.47 () {56 vars in 1074 bytes} [Sun Aug 21 01:23:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:23:15.005514 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writing DB stats nova_cell0:SELECT=4 {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:23:15.011173 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writing DB stats nova_cell1:SELECT=4 {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:23:15.011173 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG dbcounter [-] [106140] Writing DB stats nova_api:SELECT=8 {{(pid=106140) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:23:15.057751 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-1b7c27ff-13ef-4147-b5bc-2ee88150fbaf admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:23:15.060764 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 21 01:23:15.060764 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: warnings.warn(msg) Aug 21 01:23:15.064849 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-1b7c27ff-13ef-4147-b5bc-2ee88150fbaf admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:23:15.064849 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-1b7c27ff-13ef-4147-b5bc-2ee88150fbaf admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:23:15.064849 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-1b7c27ff-13ef-4147-b5bc-2ee88150fbaf admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:23:15.064849 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-1b7c27ff-13ef-4147-b5bc-2ee88150fbaf admin admin] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:23:15.064849 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-1b7c27ff-13ef-4147-b5bc-2ee88150fbaf admin admin] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:23:15.065293 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-1b7c27ff-13ef-4147-b5bc-2ee88150fbaf admin admin] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:23:15.089495 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-1b7c27ff-13ef-4147-b5bc-2ee88150fbaf admin admin] 10.222.0.47 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.088931 Aug 21 01:23:15.090048 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 38/74] 10.222.0.47 () {60 vars in 1193 bytes} [Sun Aug 21 01:23:14 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:23:15.930102 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: DEBUG nova.api.openstack.wsgi [None req-1c98c93c-5e7b-4bc9-91f0-998b15ac9989 admin admin] Calling method '>' {{(pid=106141) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:23:15.931098 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: INFO nova.api.openstack.requestlog [None req-1c98c93c-5e7b-4bc9-91f0-998b15ac9989 admin admin] 10.222.0.47 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001435 Aug 21 01:23:15.931498 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: [pid: 106141|app: 0|req: 37/75] 10.222.0.47 () {56 vars in 1074 bytes} [Sun Aug 21 01:23:15 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:23:16.135246 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG nova.api.openstack.wsgi [None req-bf0fdd08-0e33-4c48-8f86-feaaffb5474a admin admin] Calling method '>' {{(pid=106140) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:23:16.137617 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 21 01:23:16.137617 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: warnings.warn(msg) Aug 21 01:23:16.138770 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-bf0fdd08-0e33-4c48-8f86-feaaffb5474a admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:23:16.139246 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-bf0fdd08-0e33-4c48-8f86-feaaffb5474a admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:23:16.139692 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-bf0fdd08-0e33-4c48-8f86-feaaffb5474a admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:23:16.140448 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-bf0fdd08-0e33-4c48-8f86-feaaffb5474a admin admin] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:23:16.140895 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-bf0fdd08-0e33-4c48-8f86-feaaffb5474a admin admin] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:23:16.141308 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: DEBUG oslo_concurrency.lockutils [None req-bf0fdd08-0e33-4c48-8f86-feaaffb5474a admin admin] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106140) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:23:16.164018 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: INFO nova.api.openstack.requestlog [None req-bf0fdd08-0e33-4c48-8f86-feaaffb5474a admin admin] 10.222.0.47 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.094397 Aug 21 01:23:16.164597 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: [pid: 106140|app: 0|req: 39/76] 10.222.0.47 () {60 vars in 1193 bytes} [Sun Aug 21 01:23:16 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:24:15.102520 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Aug 21 01:24:15.103400 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: Sun Aug 21 01:24:15 2022 - graceful shutdown triggered... Aug 21 01:24:15.103843 dv-cinder-master-3-iscsi devstack@n-api.service[106140]: Gracefully killing worker 1 (pid: 106140)... Aug 21 01:24:15.104840 dv-cinder-master-3-iscsi devstack@n-api.service[106141]: Gracefully killing worker 2 (pid: 106141)... Aug 21 01:24:16.103944 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: worker 1 buried after 1 seconds Aug 21 01:24:16.104572 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: worker 2 buried after 1 seconds Aug 21 01:24:16.104572 dv-cinder-master-3-iscsi devstack@n-api.service[106139]: goodbye to uWSGI. Aug 21 01:24:16.105678 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Aug 21 01:24:16.106275 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Aug 21 01:24:16.109780 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Aug 21 01:24:16.124370 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Aug 21 01:24:16.127445 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Aug 21 01:24:16 2022] *** Aug 21 01:24:16.127445 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 21 01:24:16.127445 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 21 01:24:16.127445 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: nodename: dv-cinder-master-3-iscsi Aug 21 01:24:16.127445 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: machine: x86_64 Aug 21 01:24:16.127445 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: clock source: unix Aug 21 01:24:16.127445 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: pcre jit disabled Aug 21 01:24:16.128019 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: detected number of CPU cores: 4 Aug 21 01:24:16.128019 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: current working directory: / Aug 21 01:24:16.128019 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: detected binary path: /usr/bin/uwsgi-core Aug 21 01:24:16.128019 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: your processes number limit is 47680 Aug 21 01:24:16.128019 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: your memory page size is 4096 bytes Aug 21 01:24:16.128019 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: detected max file descriptor number: 2048 Aug 21 01:24:16.128019 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: lock engine: pthread robust mutexes Aug 21 01:24:16.128019 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: thunder lock: enabled Aug 21 01:24:16.128019 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Aug 21 01:24:16.128619 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 21 01:24:16.169610 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: Python main interpreter initialized at 0x5649f8e8e9c0 Aug 21 01:24:16.169610 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: python threads support enabled Aug 21 01:24:16.169610 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: your server socket listen backlog is limited to 100 connections Aug 21 01:24:16.169610 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: your mercy for graceful operations on workers is 90 seconds Aug 21 01:24:16.170237 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: mapped 403077 bytes (393 KB) for 2 cores Aug 21 01:24:16.170237 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: *** Operational MODE: preforking *** Aug 21 01:24:16.170481 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: *** uWSGI is running in multiple interpreter mode *** Aug 21 01:24:16.170481 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: spawned uWSGI master process (pid: 121261) Aug 21 01:24:16.170598 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Aug 21 01:24:16.173716 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: spawned uWSGI worker 1 (pid: 121276, cores: 1) Aug 21 01:24:16.173716 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: spawned uWSGI worker 2 (pid: 121277, cores: 1) Aug 21 01:24:16.173716 dv-cinder-master-3-iscsi devstack@n-api.service[121261]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 21 01:24:16.840143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 21 01:24:16.855532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 21 01:24:18.005516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 21 01:24:18.005516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: from cryptography.utils import int_from_bytes Aug 21 01:24:18.006176 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 21 01:24:18.006176 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: from cryptography.utils import int_from_bytes Aug 21 01:24:18.021480 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 21 01:24:18.021480 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: from cryptography.utils import int_from_bytes Aug 21 01:24:18.022080 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 21 01:24:18.022080 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: from cryptography.utils import int_from_bytes Aug 21 01:24:20.454325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 21 01:24:20.454855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 21 01:24:20.454962 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 21 01:24:20.455323 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 21 01:24:20.455408 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 21 01:24:20.455493 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 21 01:24:20.455811 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 21 01:24:20.455922 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 21 01:24:20.456010 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 21 01:24:20.456124 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.456320 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 21 01:24:20.456444 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.456684 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.456786 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.457024 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.457125 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.457352 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.457548 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.457707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.457866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.458166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.458491 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.458713 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.458830 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.459141 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.459480 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.459626 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.459748 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.460076 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.460405 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.460530 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.460749 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.461065 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.461252 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.461384 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.461725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.461853 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.462038 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.462182 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.462389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.462548 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.462710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.462894 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.463251 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.463439 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.463536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.463834 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.463942 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.464300 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.464599 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.464710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.465066 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.465310 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.465446 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.465710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.466045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.466369 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.466572 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.466691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.467015 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.467293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.467414 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.467745 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.467867 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.468068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.482048 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.482182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.482269 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.482352 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.482443 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.482523 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.482611 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.482684 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.482762 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.482834 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.482924 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.482997 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.483079 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.483151 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.483236 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.483316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.483396 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.483468 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.483545 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.483617 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.483687 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.483770 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.483850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.483932 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.484011 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.484090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.484837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.484937 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.485021 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.485094 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.485165 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.485280 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.485381 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.485466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.485466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.485604 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.485681 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.485761 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.485859 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.485947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486020 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486097 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486169 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486250 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486334 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486566 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486722 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486795 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486874 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.486946 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487015 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487270 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487415 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487504 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487583 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487664 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487744 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487825 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487905 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487985 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.487985 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.488734 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.488903 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.489060 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.489213 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.489387 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.489545 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.489692 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.489865 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.490019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.490170 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.490324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.490477 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.490640 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.47 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.490807 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.490962 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.47 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.491139 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.491547 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.491725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.491887 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.492041 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.492231 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.493251 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.493424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.493572 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.493719 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.493884 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.494039 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.494196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.494405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.495143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.495303 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.495448 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.495601 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.495774 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.47 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.495985 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.496178 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.47 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.496330 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.496473 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.496637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.496796 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.497013 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.497671 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.497828 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.497982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.498127 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.498331 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.498482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.498626 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.498780 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.498924 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.499088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.499246 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.499390 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.499534 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.499677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.499830 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.499976 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.500212 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.500377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.500530 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.500681 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.500833 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.500988 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.501145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.501290 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.501433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.501576 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.501719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.501875 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.502027 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.502173 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.502316 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.502480 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.503641 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.503845 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.503993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.504299 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.504463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.504463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.504786 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.504950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.505095 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.505463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.505621 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.505774 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.506041 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.506192 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.506348 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.506502 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.506675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.506829 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.506982 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.507127 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.507285 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.507447 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.507604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.507750 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.507904 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.508082 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.508276 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.508487 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.508634 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.508790 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.508942 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.509096 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.509248 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.509393 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.509536 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.509686 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.509839 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.510065 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.510216 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.510369 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.510514 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.510676 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.510836 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.510981 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.511132 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.511456 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.511608 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.511733 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.511846 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.511963 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.512142 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.516535 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.516714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.516875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.517035 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.517191 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.517338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.517481 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.517633 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.517789 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.517946 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.518091 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.518242 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.518473 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.518622 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.518775 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.518947 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.519106 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.519251 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.519404 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.519549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.519702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.519892 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.520050 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.520050 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.520417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.520577 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.520730 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.520875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.521032 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.521202 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.521347 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.521504 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.521650 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.521800 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.521952 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.522098 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 21 01:24:20.522242 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.522432 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.522581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.522725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.522875 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.522875 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.524185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.524358 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.524547 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.524697 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.524841 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.524994 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.525152 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.525306 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.525458 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.526235 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.526387 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.526634 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.526811 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.527008 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.527167 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.527326 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.527485 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.527485 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.527782 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.527938 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.528082 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.528296 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.528447 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.528661 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.528818 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.528973 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.529118 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.529272 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.529417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.529561 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.530912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.531073 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.531229 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.531386 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.531533 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.531734 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.531893 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.532050 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.532241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.532444 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.532636 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.532785 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.532940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.533085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.533742 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.533742 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.534248 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.534521 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.534683 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.534683 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.534980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.534980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.535263 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.535263 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.535554 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.535706 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.535852 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.535996 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.536159 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.536455 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.536614 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.536770 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.536938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.537100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.537250 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.537409 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.537585 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.537924 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.538084 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.538230 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.538375 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.538528 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.543296 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.544339 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.544519 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.544692 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.544844 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.544844 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.545397 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.545556 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.545711 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.545866 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.546508 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.546670 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.546827 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.547120 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.547276 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.547276 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.547974 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.548128 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.548252 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.548409 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.548530 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.548643 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.550906 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.551066 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.551222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.551376 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.551573 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.551732 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.552033 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.552257 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.552408 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.552597 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.552746 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.552899 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.553053 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.553198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.553341 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.553485 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.553630 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.554479 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.554643 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.554789 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.554952 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.555098 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.555250 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.555414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.555603 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.555764 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.555987 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.556313 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.556467 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.556707 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.556913 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.557069 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.557235 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.557389 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.557544 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.557715 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.557959 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.558111 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.558262 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.558425 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.558587 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.558732 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.558884 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.559081 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.559241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.559399 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.559553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.559739 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.559890 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.560034 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.560201 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.560347 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.560508 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.560660 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.560862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.561007 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.561152 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.561310 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.561466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.561620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.561773 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.561926 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.562071 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.562256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.562416 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.562568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.562721 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.562866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.563018 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.563171 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.563327 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.563479 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.563633 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.563819 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.564267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.564780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.564940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.565086 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.565239 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.565384 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.565537 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.565690 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.565842 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.565998 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.566370 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.566621 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.566785 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.566938 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.567085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.567238 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.567401 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.568390 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.568573 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.568736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.568937 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.569088 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.569302 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.569495 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.569495 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.569793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.569939 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.570092 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.570244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.570405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.570405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.570728 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.570728 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.571019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.571172 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.571344 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.571497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.571680 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.571837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.571983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.572201 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.572389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.572561 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.572714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.572859 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.573003 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.573161 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.573328 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.573501 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.573696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.573861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.574022 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.574462 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.574969 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.575129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.575296 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.575495 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.575651 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.575849 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.576005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.576198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.576400 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.576563 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.576831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.577029 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.577192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.577351 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.577536 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.577782 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.577976 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.578189 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.578494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.578659 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.578825 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.578987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.579147 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.579354 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.579511 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.579700 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.579700 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.580029 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.580234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.580388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.580538 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.580698 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.580864 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.581027 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.581192 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.581344 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.581502 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.581688 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.581843 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.581994 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.582156 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.582308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.582458 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.582608 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.582757 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.582908 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.583154 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.583308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.583467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.583619 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.583802 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.583968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.584166 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.584576 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.584757 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.585238 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.585406 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.585573 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.585724 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.585921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.586086 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.586239 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.586388 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.586548 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.586710 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.586885 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.587134 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.587290 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.587449 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.587610 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.587795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.588638 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.588842 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.589008 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.589171 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.589345 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.589554 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.589706 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.589857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.590014 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.590184 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.590338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.590496 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.590648 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.590811 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.590977 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.591142 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.591302 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.591461 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.591652 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.591808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.591966 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.592188 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.592496 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.592743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.592911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.593071 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.593451 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.593611 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.593763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.593920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.594132 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.594555 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.594877 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.596035 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.596352 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.596586 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.596812 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.597029 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.597312 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.597483 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.597645 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.597806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.597958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.598108 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.598267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.598525 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.598707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.598928 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.599090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.599223 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.599336 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.599493 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.599620 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.599741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.599863 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.599993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.600161 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.602421 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.602594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.602755 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.602908 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.603073 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.603267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.603455 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.603646 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.604385 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.604738 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.605074 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.605568 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.605728 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.605889 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.606050 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.606257 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.606426 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.606741 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.606903 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.607063 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.607227 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.607494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.607661 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.607661 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.607970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.608235 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.608394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.608579 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.608766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.608940 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.609193 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.609351 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.609510 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.609766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.609923 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.610084 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.610240 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.610390 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.610550 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.610701 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.610883 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.611048 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.611218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.611218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.611514 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.611514 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.611514 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.611977 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.612165 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.612434 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.612598 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.612759 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.612955 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.613120 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.613274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.613426 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.613578 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.613858 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.614026 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.614186 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.614339 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.614490 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.614668 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.614930 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.615232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.615691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.615860 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.615860 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.616217 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.616384 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.616537 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.616700 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.616861 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.617053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.617969 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.618134 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.618297 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.618458 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.618612 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.618770 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.618933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.619092 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.619376 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.619535 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.619742 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.619900 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.620062 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.620268 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.620423 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.620583 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.620805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.620969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.621155 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.621312 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.621462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.621614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.621788 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.621990 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.622154 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.622308 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.622467 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.622619 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.622778 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.623012 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.623012 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.623313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.623551 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.623705 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.623877 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.624221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.624396 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.624590 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.624754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.624933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.625351 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.626553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.626719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.626915 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.627073 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.627226 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.627639 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.627806 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.628088 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.628300 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.628508 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.628676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.628837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.629024 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.629191 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.629489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.629659 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.629867 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.630067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.630238 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.630399 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.630552 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.630705 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.630857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.631050 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.631217 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.631380 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.631532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.631702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.631864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.632025 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.632212 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.632463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.632626 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.632778 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.632937 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.633125 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.633298 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.633459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.633620 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.633780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.633941 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.634101 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.634268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.634444 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.634757 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.634941 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.635099 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.635538 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.636008 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.636215 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.636385 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.636628 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.636795 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.636956 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.637118 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.637279 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.637485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.637651 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.637812 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.638261 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.638431 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.638587 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.638747 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.639128 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.639337 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.639532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.639801 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.640002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.640724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.641051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.641231 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.641403 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.641403 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.641713 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.641866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.642017 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.642168 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.642168 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.642582 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.642744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.642904 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.643063 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.643242 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.643435 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.643601 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.643757 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.643910 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.644061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.644295 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.644457 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.644616 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.644786 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.645026 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.645185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.645346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.645709 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.646085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.646276 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.646441 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.646621 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.646861 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.647021 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.647182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.647335 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.647530 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.647697 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.647858 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.648022 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.648207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.648364 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.648616 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.648787 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.648949 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.649101 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.649252 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.649402 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.649561 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.649727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.650016 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.650187 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.650352 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.651035 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.651260 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.651426 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.651624 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.651782 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.651942 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.652095 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.652332 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.652485 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.652637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.652789 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.652940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.653103 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.653103 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.653409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.653560 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.653747 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.653905 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.654064 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.654226 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.654412 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.654564 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.654715 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.654866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.654866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.655296 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.655463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.655634 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.655935 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.656272 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.656456 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.656640 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.656801 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.657051 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.657209 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.657370 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.657532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.657693 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.657856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.658016 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.658169 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.658321 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.658483 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.658659 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.658909 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.659067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.659230 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.659383 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.659718 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.659879 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.660204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.660366 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.661429 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.661605 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.661796 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.661955 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.662107 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.662260 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.662411 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.662574 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.662743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.662906 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.663058 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.663223 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.663375 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.663540 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.663701 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.663854 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.664005 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.664282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.664449 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.664612 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.664765 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.664918 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.665069 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.665229 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.665402 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.665566 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.665719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.665719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.666925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.667171 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.667340 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.667340 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.668113 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.668304 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.668612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.668612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.669126 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.669224 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.669311 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.669396 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.669480 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.669573 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.669739 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.669827 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.669922 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.670008 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.670092 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.670235 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.670358 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.670452 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.670538 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.670623 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.670715 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.670815 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.670909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.670995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.671083 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.671167 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.671252 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.671368 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.671471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.671566 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.671659 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.671745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.671830 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.671830 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.672003 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.672089 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.672221 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.672221 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.672401 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.672487 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.672572 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.672663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.672763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.672857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.672952 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.673046 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.673139 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.673232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.673330 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.673433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.673520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.673611 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.673703 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.673792 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.673886 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.673971 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.674064 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.674157 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.674250 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.674342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.674437 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.674531 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.674624 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.674710 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.674795 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.674879 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.674976 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.675062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.675148 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.675148 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.675310 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.675396 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.675495 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.675495 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.675653 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.675737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.675849 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.675849 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.676133 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.676133 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.676336 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.676425 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.676509 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.676606 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.676699 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.676793 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.676895 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.676990 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.677077 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.677191 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.677288 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.677381 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.677467 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.677559 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.677662 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.677749 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.677862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.677966 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.678060 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.678145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.678238 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.678325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.678416 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.678501 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.678501 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.678655 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.678748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.678834 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.678926 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.679019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.679121 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.679207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.679308 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.679401 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.679494 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.679588 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.679680 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.679766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.679862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.679957 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.680043 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.680154 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.680259 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.680355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.680448 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.680535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.680627 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.680722 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.680820 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.680915 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.681001 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.681086 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.681170 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.681262 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.681348 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.681441 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.681535 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.681629 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.681771 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.681869 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.681971 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.682058 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.682142 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.682225 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.682309 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.682400 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.682579 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.682675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.682768 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.682864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.682957 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.683042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.683135 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.683220 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.683304 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.683388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.683479 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.683564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.683657 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.683749 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.683849 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.683935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.684027 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.684142 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.684230 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.684314 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.684409 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.684503 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.684589 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.684682 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.684776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.684861 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.684953 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.685052 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.685146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.685239 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.685327 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.685410 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.685501 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.685598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.685683 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.685776 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.685867 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.685960 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.686052 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.686191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.686295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.686390 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.686478 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.686572 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.686572 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.686728 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.686820 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.686913 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.686999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.687079 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.687173 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.687268 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.687364 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.687456 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.687541 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.687625 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.687716 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.687807 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.687906 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.687999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.688084 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.688284 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.688369 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.688453 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.688554 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.688641 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.688725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.688816 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.688901 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.688901 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.689069 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.689153 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.689244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.689244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.689408 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.689501 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.689595 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.689681 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.689773 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.689859 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.689943 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.690039 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.690125 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.690209 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.690293 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.690384 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.690483 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.690568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.690664 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.690758 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.690844 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.690935 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.691033 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.691126 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.691212 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.691303 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.691388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.691473 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.691566 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.691651 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.691735 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.691825 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.691910 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.692002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.692088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.692203 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.692292 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.692384 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.692547 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.692689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.692828 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.692967 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.693100 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.693232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.693371 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.693502 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.693640 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.693770 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.693900 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.694030 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.694174 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.694305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.694305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.694552 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.694681 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.694810 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.694948 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.695081 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.695210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.695347 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.695485 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.695630 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.695761 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.695899 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.696029 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.696187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.696312 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.696455 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.696593 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.696733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.696863 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.697000 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.697130 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.697260 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.697531 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.697668 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.697805 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.697936 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.697936 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.698199 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.698331 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.698467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.698467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.698722 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.698852 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.698989 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.699124 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.699264 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.699395 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.699514 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.699651 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.699783 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.699920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.700059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.700213 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.700359 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.700494 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.700632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.700762 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.700899 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.701030 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.701030 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.701293 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.701424 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.701560 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.701703 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.701837 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.701976 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.702108 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.702246 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.702378 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.702378 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.702625 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.702831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.702964 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.703094 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.703224 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.703353 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.704081 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.704081 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.704394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.704394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.704642 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.704642 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.704906 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.705045 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.705183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.705321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.705461 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.705599 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.705744 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.705882 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.706013 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.706143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.706281 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.706401 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.706542 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.706682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.706820 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.706958 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.707096 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.707240 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.707360 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.707488 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.707616 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.707745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.707882 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.708025 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.708246 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.708379 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.708510 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.708639 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.708769 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.708912 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.709042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.709181 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.709319 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.709458 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.709588 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.709727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.709860 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.709997 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.710112 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.710244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.710388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.710521 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.710651 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.710779 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.710909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.711046 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.711046 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.711302 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.711432 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.711569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.711700 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.711836 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.711976 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.712122 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.712264 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.712403 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.712533 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.712663 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.712806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.712945 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.713580 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.713719 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.713857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.714003 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.714134 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.714272 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.714272 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.714525 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.714662 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.714802 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.714924 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.715037 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.715037 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.715282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.715411 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.715548 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.715687 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.715817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.715817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.716062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.716312 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.716550 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.716712 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.716844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.716982 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.717114 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.717254 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.717393 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.717393 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.717644 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.717776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.717913 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.718052 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.718191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.718322 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.718459 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.718590 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.718720 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.718864 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.718996 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.719126 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.719268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.719400 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.719529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.719672 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.719793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.719929 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.720060 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.720202 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.720351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.720495 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.720633 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.720764 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.720894 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.721020 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.721163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.721304 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.721435 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.721565 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.721694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.721823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.721962 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.722101 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.722232 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.722946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.723085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.723225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.723357 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.723488 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.723626 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.723764 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.724551 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.724702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.724835 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.724986 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.725119 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.725251 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.725389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.725536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.725667 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.725798 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.725927 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.726069 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.726209 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.726341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.726471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.726603 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.726726 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.726871 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.726871 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.727137 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.727137 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.727399 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.727537 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.727676 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.727808 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.727938 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.728075 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.728223 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.728337 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.728466 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.728627 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.729277 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.729411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.729542 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.729683 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.729824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.729966 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.730107 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.730243 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.730384 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.730515 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.730735 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.730878 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.731010 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.731130 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.731259 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.731399 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.731538 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.731670 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.731800 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.731937 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.732076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.732231 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.732371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.732503 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.732640 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.732771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.732901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.733045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.733185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.733324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.733467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.733598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.733727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.733866 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.733997 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.734135 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.734265 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.734402 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.734550 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.734682 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.734820 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.734952 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.735082 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.735220 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.735361 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.735498 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.735629 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.735629 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.735882 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.736021 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.736218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.736351 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.736482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.736612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.736739 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.736866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.736996 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.737116 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.737230 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.737342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.737453 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.737596 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.737717 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.737845 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.737982 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.738245 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.738338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.738437 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.738525 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.738617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.738711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.738806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.738904 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.739002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.739097 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.739192 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.739288 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.739374 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.739510 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.739616 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.739704 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.739799 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.739885 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.739969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.740063 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.740234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.740334 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.740418 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.740504 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.740586 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.740729 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.740821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.740912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.740998 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.741082 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.741178 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.741313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.741407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.741501 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.741600 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.741687 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.741784 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.741871 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.741967 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.742068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.742163 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.742246 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.742340 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.742427 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.742517 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.742610 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.742735 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.742828 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.742931 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.743025 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.743111 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.743196 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.743280 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.743413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.743517 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.743605 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.743690 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.743784 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.743871 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.743965 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.744051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.744174 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.744286 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.744384 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.744471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.744565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.744659 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.744757 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.744842 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.744927 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.745022 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.745116 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.745216 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.745309 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.745395 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.745487 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.745586 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.745677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.745763 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.745907 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.745997 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.746090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.746189 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.746189 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.746354 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.746429 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.746522 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.746608 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.746608 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.746774 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.746905 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.747000 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.747138 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.747244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.747244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.747422 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.747509 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.747601 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.747687 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.747781 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.747855 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.747939 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=121277) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 21 01:24:20.748058 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.748058 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.748058 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.748058 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.748058 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.748536 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.748536 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 21 01:24:20.748735 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=121276) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 21 01:24:20.808467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO dbcounter [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Registered counter for database nova_api Aug 21 01:24:20.810869 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO dbcounter [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Registered counter for database nova_api Aug 21 01:24:20.829061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_db.sqlalchemy.engines [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121277) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:24:20.830735 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_db.sqlalchemy.engines [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121276) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:24:20.837196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writer thread running {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:24:20.837385 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writer thread running {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:24:20.894550 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.context [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),6547be85-e237-4d35-aaa7-703fe51c4e0b(cell1) {{(pid=121276) load_cells /opt/stack/nova/nova/context.py:464}} Aug 21 01:24:20.895157 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.context [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),6547be85-e237-4d35-aaa7-703fe51c4e0b(cell1) {{(pid=121277) load_cells /opt/stack/nova/nova/context.py:464}} Aug 21 01:24:20.898699 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:24:20.899169 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:24:20.899295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:24:20.899694 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:24:20.901408 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:24:20.901943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:24:20.902146 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:24:20.902593 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:24:20.902772 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:24:20.903127 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:24:20.904587 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:24:20.905140 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:24:20.906596 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO dbcounter [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Registered counter for database nova_cell0 Aug 21 01:24:20.907309 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO dbcounter [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Registered counter for database nova_cell0 Aug 21 01:24:20.910597 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO dbcounter [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Registered counter for database nova_cell1 Aug 21 01:24:20.911660 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO dbcounter [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Registered counter for database nova_cell1 Aug 21 01:24:20.918853 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_db.sqlalchemy.engines [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121276) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:24:20.920366 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_db.sqlalchemy.engines [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121277) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:24:20.921608 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_db.sqlalchemy.engines [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121277) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:24:20.923978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_db.sqlalchemy.engines [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121276) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:24:20.928206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writer thread running {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:24:20.930276 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writer thread running {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:24:20.931441 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writer thread running {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:24:20.943718 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writer thread running {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:24:21.079937 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO dbcounter [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Registered counter for database nova_cell0 Aug 21 01:24:21.081654 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO dbcounter [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Registered counter for database nova_cell0 Aug 21 01:24:21.089656 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_db.sqlalchemy.engines [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121276) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:24:21.089855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_db.sqlalchemy.engines [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121277) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 21 01:24:21.094687 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writer thread running {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:24:21.095300 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writer thread running {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 21 01:24:21.416835 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: WARNING keystonemiddleware.auth_token [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 21 01:24:21.417024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: WARNING keystonemiddleware.auth_token [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 21 01:24:21.424451 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: WARNING keystonemiddleware.auth_token [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 21 01:24:21.424606 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: WARNING keystonemiddleware.auth_token [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 21 01:24:21.780137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: WARNING keystonemiddleware.auth_token [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 21 01:24:21.780773 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: WARNING keystonemiddleware.auth_token [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 21 01:24:21.786455 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: WARNING keystonemiddleware.auth_token [None req-a158499d-5a93-482d-8d34-f1d0177d8a93 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 21 01:24:21.786594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: WARNING keystonemiddleware.auth_token [None req-41cab75e-4147-405c-86de-e53e9f6befe2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 21 01:24:21.791989 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5649f8e8e9c0 pid: 121277 (default app) Aug 21 01:24:21.792187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5649f8e8e9c0 pid: 121276 (default app) Aug 21 01:24:57.453758 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=3 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:24:57.454618 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=3 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:24:57.455486 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=3 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:24:57.456245 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=3 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:24:58.395058 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=3 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:24:58.396654 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=3 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:24:58.397531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=3 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:24:58.398417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=3 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:24:58.850041 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3fbb7aca-f240-42fe-85f9-d3d52f448be2 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Action: 'create', calling method: .version_select at 0x7f9da947b5e0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1505450106"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:24:59.750126 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6ee23bec-d6be-427a-9557-1fbe12dff7e4 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Action: 'create', calling method: .version_select at 0x7f9da947c700>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1653039741", "metadata": {"Type": "work"}}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6ee23bec-d6be-427a-9557-1fbe12dff7e4 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Request method failure captured: Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 122 Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYlqF1627tIENW_nmd634LIdFPxsB2HLRjekTyvWOsTCNG21hku41I3QCRZ8RPDiRI6f367tuGVBqTjEdVwmzy6Ry3j6DXTtDYboxsfHEAAOOF24uHFd8eMQ5sC-F1l_yYN8HoIWA5ztwSzst_hsGYlAYcE8AJ67-e5PKePoBh7VgoU Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 0cdbd53cdbbf4cc3a1bc436bd329b60f Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-VolumesNegativeTest-1501754625 Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:24:59.756599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-VolumesNegativeTest-1501754625 Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 0cdbd53cdbbf4cc3a1bc436bd329b60f Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-VolumesNegativeTest-1501754625 Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-VolumesNegativeTest-1501754625-project Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a5c6871a2f0046aeb69b82e0c1529376 Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-VolumesNegativeTest-1501754625-project Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1653039741", "metadata": {"Type": "work"}}} Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: Aug 21 01:24:59.763188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1653039741', 'metadata': {'Type': 'work'}}}} Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise error Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi '#$%' Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:24:59.764471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 21 01:24:59.765817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 21 01:24:59.767453 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 21 01:24:59.767453 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.767453 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6ee23bec-d6be-427a-9557-1fbe12dff7e4 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:24:59.767453 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6ee23bec-d6be-427a-9557-1fbe12dff7e4 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] 10.222.0.47 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 1.372848 Aug 21 01:24:59.767453 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 1/1] 10.222.0.47 () {60 vars in 1156 bytes} [Sun Aug 21 01:24:58 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 1375 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 21 01:24:59.781658 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-785aec7b-2724-493a-bfd2-09b367323bef tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Action: 'create', calling method: .version_select at 0x7f9da947c790>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-933059501", "metadata": {"Type": "work"}}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-785aec7b-2724-493a-bfd2-09b367323bef tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Request method failure captured: Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 119 Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYlqF1627tIENW_nmd634LIdFPxsB2HLRjekTyvWOsTCNG21hku41I3QCRZ8RPDiRI6f367tuGVBqTjEdVwmzy6Ry3j6DXTtDYboxsfHEAAOOF24uHFd8eMQ5sC-F1l_yYN8HoIWA5ztwSzst_hsGYlAYcE8AJ67-e5PKePoBh7VgoU Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 0cdbd53cdbbf4cc3a1bc436bd329b60f Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-VolumesNegativeTest-1501754625 Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:24:59.787864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-VolumesNegativeTest-1501754625 Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 0cdbd53cdbbf4cc3a1bc436bd329b60f Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-VolumesNegativeTest-1501754625 Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-VolumesNegativeTest-1501754625-project Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a5c6871a2f0046aeb69b82e0c1529376 Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-VolumesNegativeTest-1501754625-project Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-933059501", "metadata": {"Type": "work"}}} Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: Aug 21 01:24:59.789324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-933059501', 'metadata': {'Type': 'work'}}}} Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise error Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi '0' Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:24:59.790166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 21 01:24:59.791711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 21 01:24:59.793006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 21 01:24:59.793006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.793006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-785aec7b-2724-493a-bfd2-09b367323bef tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:24:59.793006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-785aec7b-2724-493a-bfd2-09b367323bef tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] 10.222.0.47 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.018368 Aug 21 01:24:59.793006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 2/2] 10.222.0.47 () {60 vars in 1156 bytes} [Sun Aug 21 01:24:59 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 21 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 21 01:24:59.808639 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8af3f00f-31a1-4eca-8c05-eaeb9a14ae79 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Action: 'create', calling method: .version_select at 0x7f9da93b7280>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1656038048", "metadata": {"Type": "work"}}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8af3f00f-31a1-4eca-8c05-eaeb9a14ae79 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Request method failure captured: Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 119 Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYlqF1627tIENW_nmd634LIdFPxsB2HLRjekTyvWOsTCNG21hku41I3QCRZ8RPDiRI6f367tuGVBqTjEdVwmzy6Ry3j6DXTtDYboxsfHEAAOOF24uHFd8eMQ5sC-F1l_yYN8HoIWA5ztwSzst_hsGYlAYcE8AJ67-e5PKePoBh7VgoU Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 0cdbd53cdbbf4cc3a1bc436bd329b60f Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-VolumesNegativeTest-1501754625 Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:24:59.815789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-VolumesNegativeTest-1501754625 Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 0cdbd53cdbbf4cc3a1bc436bd329b60f Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-VolumesNegativeTest-1501754625 Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-VolumesNegativeTest-1501754625-project Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a5c6871a2f0046aeb69b82e0c1529376 Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-VolumesNegativeTest-1501754625-project Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1656038048", "metadata": {"Type": "work"}}} Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: Aug 21 01:24:59.818068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1656038048', 'metadata': {'Type': 'work'}}}} Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise error Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi '' Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:24:59.819486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 21 01:24:59.821433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 21 01:24:59.824091 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 21 01:24:59.824091 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:24:59.824995 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8af3f00f-31a1-4eca-8c05-eaeb9a14ae79 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:24:59.825907 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8af3f00f-31a1-4eca-8c05-eaeb9a14ae79 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] 10.222.0.47 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.025467 Aug 21 01:24:59.826797 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 3/3] 10.222.0.47 () {60 vars in 1156 bytes} [Sun Aug 21 01:24:59 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 30 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Aug 21 01:24:59.837704 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-90f539b0-0c7f-445d-a623-2f2d61842217 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Calling method '.version_select at 0x7f9da93b7c10>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-90f539b0-0c7f-445d-a623-2f2d61842217 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] HTTP exception thrown: Volume tempest-invalid-1743043591 could not be found. Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-90f539b0-0c7f-445d-a623-2f2d61842217 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Request method failure captured: Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1743043591 HTTP/1.1 Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYlqF1627tIENW_nmd634LIdFPxsB2HLRjekTyvWOsTCNG21hku41I3QCRZ8RPDiRI6f367tuGVBqTjEdVwmzy6Ry3j6DXTtDYboxsfHEAAOOF24uHFd8eMQ5sC-F1l_yYN8HoIWA5ztwSzst_hsGYlAYcE8AJ67-e5PKePoBh7VgoU Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 0cdbd53cdbbf4cc3a1bc436bd329b60f Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-VolumesNegativeTest-1501754625 Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:25:01.673042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-VolumesNegativeTest-1501754625 Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 0cdbd53cdbbf4cc3a1bc436bd329b60f Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-VolumesNegativeTest-1501754625 Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-VolumesNegativeTest-1501754625-project Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a5c6871a2f0046aeb69b82e0c1529376 Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-VolumesNegativeTest-1501754625-project Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Volume tempest-invalid-1743043591 could not be found. Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': 'tempest-invalid-1743043591'} Aug 21 01:25:01.686766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1743043591 could not be found. (HTTP 404) (Request-ID: req-8a946172-800d-4b68-bb4f-26f981c0d4ce) Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:01.691333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 21 01:25:01.697133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1743043591 could not be found. Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:25:01.702804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1743043591 could not be found. Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1743043591 could not be found. Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-90f539b0-0c7f-445d-a623-2f2d61842217 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Returning 404 to user: Volume tempest-invalid-1743043591 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-90f539b0-0c7f-445d-a623-2f2d61842217 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] 10.222.0.47 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1743043591" status: 404 len: 99 microversion: 2.1 time: 1.374545 Aug 21 01:25:01.709117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 4/4] 10.222.0.47 () {58 vars in 1218 bytes} [Sun Aug 21 01:24:59 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1743043591 => generated 99 bytes in 1376 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Aug 21 01:25:01.714381 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5e46fe88-1dda-42f6-8711-6b6957423cc2 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] 10.222.0.47 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.006418 Aug 21 01:25:01.714381 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 5/5] 10.222.0.47 () {58 vars in 1140 bytes} [Sun Aug 21 01:25:01 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 14 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Aug 21 01:25:01.714381 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d79fd572-d5ce-49e6-9069-09c3e9e57bd7 tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003945 Aug 21 01:25:01.714381 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 6/6] 10.222.0.47 () {58 vars in 1137 bytes} [Sun Aug 21 01:25:01 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 7 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Aug 21 01:25:01.714381 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3828171f-d504-44f8-bae3-935c9b7bd5ff tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Calling method '.version_select at 0x7f9da92eb550>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:01.898234 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-3828171f-d504-44f8-bae3-935c9b7bd5ff tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] HTTP exception thrown: Volume 5cade5ca-a7c6-4f41-a87d-d7a8cc4cc65a could not be found. Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3828171f-d504-44f8-bae3-935c9b7bd5ff tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Request method failure captured: Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: DELETE /compute/v2.1/os-volumes/5cade5ca-a7c6-4f41-a87d-d7a8cc4cc65a HTTP/1.1 Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYlqF1627tIENW_nmd634LIdFPxsB2HLRjekTyvWOsTCNG21hku41I3QCRZ8RPDiRI6f367tuGVBqTjEdVwmzy6Ry3j6DXTtDYboxsfHEAAOOF24uHFd8eMQ5sC-F1l_yYN8HoIWA5ztwSzst_hsGYlAYcE8AJ67-e5PKePoBh7VgoU Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 0cdbd53cdbbf4cc3a1bc436bd329b60f Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-VolumesNegativeTest-1501754625 Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:25:01.900857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-VolumesNegativeTest-1501754625 Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 0cdbd53cdbbf4cc3a1bc436bd329b60f Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-VolumesNegativeTest-1501754625 Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-VolumesNegativeTest-1501754625-project Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a5c6871a2f0046aeb69b82e0c1529376 Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-VolumesNegativeTest-1501754625-project Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Volume 5cade5ca-a7c6-4f41-a87d-d7a8cc4cc65a could not be found. Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '5cade5ca-a7c6-4f41-a87d-d7a8cc4cc65a'} Aug 21 01:25:01.902059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 5cade5ca-a7c6-4f41-a87d-d7a8cc4cc65a could not be found. (HTTP 404) (Request-ID: req-8d6fb28c-cd49-42e4-94b5-3cba3b02bffc) Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:01.902824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 21 01:25:01.913479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 5cade5ca-a7c6-4f41-a87d-d7a8cc4cc65a could not be found. Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 21 01:25:01.914325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 5cade5ca-a7c6-4f41-a87d-d7a8cc4cc65a could not be found. Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 5cade5ca-a7c6-4f41-a87d-d7a8cc4cc65a could not be found. Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3828171f-d504-44f8-bae3-935c9b7bd5ff tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Returning 404 to user: Volume 5cade5ca-a7c6-4f41-a87d-d7a8cc4cc65a could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3828171f-d504-44f8-bae3-935c9b7bd5ff tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] 10.222.0.47 "DELETE /compute/v2.1/os-volumes/5cade5ca-a7c6-4f41-a87d-d7a8cc4cc65a" status: 404 len: 109 microversion: 2.1 time: 0.638122 Aug 21 01:25:01.916050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 7/7] 10.222.0.47 () {58 vars in 1248 bytes} [Sun Aug 21 01:25:01 2022] DELETE /compute/v2.1/os-volumes/5cade5ca-a7c6-4f41-a87d-d7a8cc4cc65a => generated 109 bytes in 644 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:25:01.920564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b4eac1e6-b6cb-4025-8c6c-f33ba9630aad tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Calling method '.version_select at 0x7f9da92eb820>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:02.282035 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3fbb7aca-f240-42fe-85f9-d3d52f448be2 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 4.833139 Aug 21 01:25:02.283255 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 1/8] 10.222.0.47 () {60 vars in 1155 bytes} [Sun Aug 21 01:24:57 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 4837 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Aug 21 01:25:02.304233 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2d7a3600-0bad-4236-88f5-7c94dd5db245 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da93514c0>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:02.314343 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-b4eac1e6-b6cb-4025-8c6c-f33ba9630aad tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] HTTP exception thrown: Volume edbfb7bb-fe8b-4a8e-8c46-63089f8a1af1 could not be found. Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b4eac1e6-b6cb-4025-8c6c-f33ba9630aad tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Request method failure captured: Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/os-volumes/edbfb7bb-fe8b-4a8e-8c46-63089f8a1af1 HTTP/1.1 Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYlqF1627tIENW_nmd634LIdFPxsB2HLRjekTyvWOsTCNG21hku41I3QCRZ8RPDiRI6f367tuGVBqTjEdVwmzy6Ry3j6DXTtDYboxsfHEAAOOF24uHFd8eMQ5sC-F1l_yYN8HoIWA5ztwSzst_hsGYlAYcE8AJ67-e5PKePoBh7VgoU Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 0cdbd53cdbbf4cc3a1bc436bd329b60f Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-VolumesNegativeTest-1501754625 Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:25:02.317105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0cdbd53cdbbf4cc3a1bc436bd329b60f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-VolumesNegativeTest-1501754625 Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 0cdbd53cdbbf4cc3a1bc436bd329b60f Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-VolumesNegativeTest-1501754625 Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-VolumesNegativeTest-1501754625-project Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a5c6871a2f0046aeb69b82e0c1529376 Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-VolumesNegativeTest-1501754625-project Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Volume edbfb7bb-fe8b-4a8e-8c46-63089f8a1af1 could not be found. Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': 'edbfb7bb-fe8b-4a8e-8c46-63089f8a1af1'} Aug 21 01:25:02.318207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume edbfb7bb-fe8b-4a8e-8c46-63089f8a1af1 could not be found. (HTTP 404) (Request-ID: req-50d203e7-7711-4a24-af6b-653e1fc7b09f) Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:02.318805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 21 01:25:02.319691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume edbfb7bb-fe8b-4a8e-8c46-63089f8a1af1 could not be found. Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 21 01:25:02.320581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:25:02.321466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume edbfb7bb-fe8b-4a8e-8c46-63089f8a1af1 could not be found. Aug 21 01:25:02.321466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:02.321466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:02.321466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:02.321466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:02.321466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:02.321466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:02.321466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:25:02.321466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:25:02.321466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume edbfb7bb-fe8b-4a8e-8c46-63089f8a1af1 could not be found. Aug 21 01:25:02.321466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:02.321466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b4eac1e6-b6cb-4025-8c6c-f33ba9630aad tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] Returning 404 to user: Volume edbfb7bb-fe8b-4a8e-8c46-63089f8a1af1 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:25:02.322394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b4eac1e6-b6cb-4025-8c6c-f33ba9630aad tempest-VolumesNegativeTest-1501754625 tempest-VolumesNegativeTest-1501754625-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/edbfb7bb-fe8b-4a8e-8c46-63089f8a1af1" status: 404 len: 109 microversion: 2.1 time: 0.413804 Aug 21 01:25:02.323490 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 8/9] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:25:01 2022] GET /compute/v2.1/os-volumes/edbfb7bb-fe8b-4a8e-8c46-63089f8a1af1 => generated 109 bytes in 415 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:25:02.640209 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2d7a3600-0bad-4236-88f5-7c94dd5db245 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee" status: 200 len: 345 microversion: 2.1 time: 0.344608 Aug 21 01:25:02.642775 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 2/10] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:25:02 2022] GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee => generated 345 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:03.655442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d165ec27-8436-400e-a41d-89b203270323 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da93b7f70>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:03.785604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d165ec27-8436-400e-a41d-89b203270323 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee" status: 200 len: 346 microversion: 2.1 time: 0.134459 Aug 21 01:25:03.786179 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 9/11] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:25:03 2022] GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee => generated 346 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:03.803305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-45594e3d-9213-4d77-b08b-d8d000aa788c tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Action: 'create', calling method: .version_select at 0x7f9da9351af0>, body: {"snapshot": {"volume_id": "af225eb8-7cc0-48cb-b0f6-d8d95cff16ee", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-469428007"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:05.439728 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-45594e3d-9213-4d77-b08b-d8d000aa788c tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 1.642169 Aug 21 01:25:05.443156 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 3/12] 10.222.0.47 () {60 vars in 1162 bytes} [Sun Aug 21 01:25:03 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 1648 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:05.454953 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7baa05fd-17f3-4ec6-838c-d7c6e20fdb54 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da92ebaf0>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:05.592862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7baa05fd-17f3-4ec6-838c-d7c6e20fdb54 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972" status: 200 len: 286 microversion: 2.1 time: 0.140910 Aug 21 01:25:05.592862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 10/13] 10.222.0.47 () {58 vars in 1251 bytes} [Sun Aug 21 01:25:05 2022] GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:05.600881 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5fb298a7-ba18-4750-a066-a67645e3e911 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da9360ca0>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:06.468147 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5fb298a7-ba18-4750-a066-a67645e3e911 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.868933 Aug 21 01:25:06.469194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 4/14] 10.222.0.47 () {58 vars in 1140 bytes} [Sun Aug 21 01:25:05 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 872 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:06.487367 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c2003b38-19a5-4f0f-bfec-b75220232204 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da92ebc10>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:06.583689 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c2003b38-19a5-4f0f-bfec-b75220232204 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972" status: 200 len: 286 microversion: 2.1 time: 0.100893 Aug 21 01:25:06.589342 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 11/15] 10.222.0.47 () {58 vars in 1251 bytes} [Sun Aug 21 01:25:06 2022] GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972 => generated 286 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:07.596624 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ae8b334b-bbbb-408a-8c87-81d0cd1dc973 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da9351b80>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:07.719617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ae8b334b-bbbb-408a-8c87-81d0cd1dc973 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972" status: 200 len: 286 microversion: 2.1 time: 0.127162 Aug 21 01:25:07.720439 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 5/16] 10.222.0.47 () {58 vars in 1251 bytes} [Sun Aug 21 01:25:07 2022] GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972 => generated 286 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:25:08.734526 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-43691483-2c04-4cd4-b4c6-20be4294ba18 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da92e7280>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:08.795794 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-43691483-2c04-4cd4-b4c6-20be4294ba18 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972" status: 200 len: 286 microversion: 2.1 time: 0.066102 Aug 21 01:25:08.797035 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 12/17] 10.222.0.47 () {58 vars in 1251 bytes} [Sun Aug 21 01:25:08 2022] GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972 => generated 286 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:09.810431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9addeb3b-516a-4ddf-8069-2ec6f7ac06bf tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da9351820>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:09.860324 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9addeb3b-516a-4ddf-8069-2ec6f7ac06bf tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972" status: 200 len: 287 microversion: 2.1 time: 0.053350 Aug 21 01:25:09.860324 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 6/18] 10.222.0.47 () {58 vars in 1251 bytes} [Sun Aug 21 01:25:09 2022] GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972 => generated 287 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:09.871236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c6b983aa-ee72-462b-ac36-c24e09c1731b tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da92eb4c0>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:10.068440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c6b983aa-ee72-462b-ac36-c24e09c1731b tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "DELETE /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972" status: 202 len: 0 microversion: 2.1 time: 0.202102 Aug 21 01:25:10.069236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 13/19] 10.222.0.47 () {58 vars in 1254 bytes} [Sun Aug 21 01:25:09 2022] DELETE /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972 => generated 0 bytes in 204 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:25:10.085217 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8b4a0d1e-655e-4041-9b41-88590eb799c5 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da947b700>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:10.127814 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8b4a0d1e-655e-4041-9b41-88590eb799c5 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972" status: 200 len: 286 microversion: 2.1 time: 0.052476 Aug 21 01:25:10.128777 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 7/20] 10.222.0.47 () {58 vars in 1251 bytes} [Sun Aug 21 01:25:10 2022] GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972 => generated 286 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:10.966362 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1847328338", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:11.139965 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4dd15679-355b-457f-bda0-2c8d8c0783a2 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da9351820>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:11.158231 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] validate_networks() for [('679614c9-0d32-43d1-9058-1d318da99722', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:25:11.244764 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4dd15679-355b-457f-bda0-2c8d8c0783a2 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972" status: 200 len: 286 microversion: 2.1 time: 0.108782 Aug 21 01:25:11.245660 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 8/21] 10.222.0.47 () {58 vars in 1251 bytes} [Sun Aug 21 01:25:11 2022] GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972 => generated 286 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:12.068401 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1107004917", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "networks": [{"uuid": "6533a9e0-1a39-42a0-8035-a9b34f9a540b"}]}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:12.359421 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.network.neutron [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] validate_networks() for [('6533a9e0-1a39-42a0-8035-a9b34f9a540b', None, None, None, None, None)] {{(pid=121277) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:25:13.316657 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:25:13.318848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:25:13.320412 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:25:13.320979 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:25:13.324496 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:25:13.368190 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:13.377389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:13.381715 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:13.383543 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:13.384234 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:13.384818 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:13.490588 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Getting quotas for project 9774c354b8b048768f2071acfb48a4bf. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:25:13.517658 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Getting quotas for user d84b6d05decb47c7b55ac782d5839c1a and project 9774c354b8b048768f2071acfb48a4bf. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:25:13.553742 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:25:13.570911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:25:13.825849 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d8246ab5-0e03-4f79-bdd4-0934b561e778 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 3.059768 Aug 21 01:25:13.826522 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 14/22] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:25:10 2022] POST /compute/v2.1/servers => generated 383 bytes in 3062 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:25:13.836992 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-32cb3375-eb63-4b30-a839-4ae6d8d62992 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da91f6670>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:14.197611 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-32cb3375-eb63-4b30-a839-4ae6d8d62992 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] HTTP exception thrown: Snapshot 5a2c2068-0c8b-4d9a-b7bc-da6002850972 could not be found. Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-32cb3375-eb63-4b30-a839-4ae6d8d62992 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Request method failure captured: Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972 HTTP/1.1 Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYlpE0IxnmNgi7UI6prU850cdhWavoXZGs4GneXBCQVthKqGCiyrTKRCpJ1wq7liWPUv1HC3awYWnDyqTyaJxPSRhA1WmuPLZ87DWYRNXNUYK0WVHUTp8zS-BmVwKiS9huGBHXrS3wXfFEkBL4JG9sIM6nXKcoxmgSLa_G4NuuY0yXY Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 1f324851bf32439880d91a81160858ab Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-283509664 Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:25:14.212598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/1f324851bf32439880d91a81160858ab"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_1f324851bf32439880d91a81160858ab"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/1f324851bf32439880d91a81160858ab"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/1f324851bf32439880d91a81160858ab"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-VolumesSnapshotsTestJSON-283509664 Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 1f324851bf32439880d91a81160858ab Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-283509664 Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-VolumesSnapshotsTestJSON-283509664-project Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: dc3e6a6484c74d2cab56e6696404817d Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-VolumesSnapshotsTestJSON-283509664-project Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Snapshot 5a2c2068-0c8b-4d9a-b7bc-da6002850972 could not be found. Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '5a2c2068-0c8b-4d9a-b7bc-da6002850972'} Aug 21 01:25:14.215970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 5a2c2068-0c8b-4d9a-b7bc-da6002850972 could not be found. (HTTP 404) (Request-ID: req-d284dcc3-1457-4f6a-9289-e0a8f0acdf4f) Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:14.220819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 21 01:25:14.222978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 5a2c2068-0c8b-4d9a-b7bc-da6002850972 could not be found. Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Aug 21 01:25:14.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:25:14.232207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 5a2c2068-0c8b-4d9a-b7bc-da6002850972 could not be found. Aug 21 01:25:14.232207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:14.232207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:14.232207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:14.232207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:14.232207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:14.232207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:14.232207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:25:14.232207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:25:14.232207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 5a2c2068-0c8b-4d9a-b7bc-da6002850972 could not be found. Aug 21 01:25:14.232207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:14.235988 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-32cb3375-eb63-4b30-a839-4ae6d8d62992 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Returning 404 to user: Snapshot 5a2c2068-0c8b-4d9a-b7bc-da6002850972 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:25:14.254767 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-32cb3375-eb63-4b30-a839-4ae6d8d62992 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972" status: 404 len: 111 microversion: 2.1 time: 0.407771 Aug 21 01:25:14.263754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 15/23] 10.222.0.47 () {58 vars in 1251 bytes} [Sun Aug 21 01:25:13 2022] GET /compute/v2.1/os-snapshots/5a2c2068-0c8b-4d9a-b7bc-da6002850972 => generated 111 bytes in 436 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:25:14.304794 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8b12c531-4580-4445-a6f6-23c3ddcae104 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:14.308990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8b12c531-4580-4445-a6f6-23c3ddcae104 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:15.143320 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8b12c531-4580-4445-a6f6-23c3ddcae104 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:15.145829 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Flavor limits 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:25:15.146514 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Image limits 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:25:15.147244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Flavor pref 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:25:15.147879 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Image pref 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:25:15.149126 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:25:15.208205 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8b12c531-4580-4445-a6f6-23c3ddcae104 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1222 microversion: 2.1 time: 0.942592 Aug 21 01:25:15.209908 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 16/24] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:14 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1222 bytes in 946 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:15.238959 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:15.246164 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:15.246803 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:15.249761 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:15.253509 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:15.257309 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1319da89-06e9-4916-a4bf-828a59dc730b tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da8187e50>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:15.259212 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:15.493216 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Getting quotas for project a1914f8232864af6ad97aebcb1636125. Resources: {'instances', 'ram', 'cores'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:25:15.522616 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Getting quotas for user bc7137d0ba634b6ab4b83de1ab47c476 and project a1914f8232864af6ad97aebcb1636125. Resources: {'instances', 'ram', 'cores'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:25:15.555564 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Going to run 1 instances... {{(pid=121277) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:25:15.560990 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121277) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:25:15.755989 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1319da89-06e9-4916-a4bf-828a59dc730b tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "DELETE /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee" status: 202 len: 0 microversion: 2.1 time: 0.523892 Aug 21 01:25:15.757600 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 17/25] 10.222.0.47 () {58 vars in 1248 bytes} [Sun Aug 21 01:25:15 2022] DELETE /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee => generated 0 bytes in 534 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:25:15.775070 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1541267683", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:15.897810 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] validate_networks() for [('679614c9-0d32-43d1-9058-1d318da99722', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:25:15.940209 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e96bd2c3-de3d-47f5-8864-5d27d1dc8ace tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.011915 Aug 21 01:25:15.941358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 9/26] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:25:11 2022] POST /compute/v2.1/servers => generated 380 bytes in 4014 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:25:16.057041 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e7402043-b81e-4a7b-bf12-8ce7a0b3780f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Action: 'create', calling method: .version_select at 0x7f9da926caf0>, body: {"keypair": {"name": "tempest-keypair-1692936143", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGwk2PjnftjYZFvnYnsbRpfMudvODOeiNoDX2yFfKih/DVHP5znlRxh3cZ77rFJH1Md2kw++DKtzWBMq0reZaJcOdPojJTDON2BiHnpWC3Vq4YSgmMkdHzhJMte0eunO6w=="}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:16.114325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-e7402043-b81e-4a7b-bf12-8ce7a0b3780f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Getting quotas for project f4a0fcea08b248fe84012875a7243761. Resources: {'key_pairs'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:25:16.126634 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-e7402043-b81e-4a7b-bf12-8ce7a0b3780f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Getting quotas for user 0cd8d822f15943a78575c3744692cdb9 and project f4a0fcea08b248fe84012875a7243761. Resources: {'key_pairs'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:25:16.267499 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e7402043-b81e-4a7b-bf12-8ce7a0b3780f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.317608 Aug 21 01:25:16.268781 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 10/27] 10.222.0.47 () {60 vars in 1159 bytes} [Sun Aug 21 01:25:15 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:16.295589 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8a334c17-8730-4fe5-9829-239a8156f2e2 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da947b700>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:16.446078 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8a334c17-8730-4fe5-9829-239a8156f2e2 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee" status: 200 len: 345 microversion: 2.1 time: 0.160221 Aug 21 01:25:16.447405 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 11/28] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:25:16 2022] GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee => generated 345 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:16.466206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-81995a87-d8bd-4ef0-bd98-a362f47e42ed tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:16.466860 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-81995a87-d8bd-4ef0-bd98-a362f47e42ed tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:16.860232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-81995a87-d8bd-4ef0-bd98-a362f47e42ed tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:16.860232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-81995a87-d8bd-4ef0-bd98-a362f47e42ed tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1215 microversion: 2.1 time: 0.267567 Aug 21 01:25:16.860232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 12/29] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:16 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1215 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:17.157029 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:25:17.157590 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:25:17.158284 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:25:17.158707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:25:17.160327 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:25:17.187374 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:17.187859 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:17.188439 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:17.189219 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:17.189727 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:17.190202 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:17.230166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Getting quotas for project 9774c354b8b048768f2071acfb48a4bf. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:25:17.242629 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Getting quotas for user d84b6d05decb47c7b55ac782d5839c1a and project 9774c354b8b048768f2071acfb48a4bf. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:25:17.262419 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:25:17.272389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:25:17.462612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4f63ccb6-2f3c-476a-901b-a49022dc125c tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da8209280>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:17.471779 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-170618ea-1191-4580-8052-8be997898aed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.706433 Aug 21 01:25:17.492737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 18/30] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:25:15 2022] POST /compute/v2.1/servers => generated 383 bytes in 1734 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:25:17.512279 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ea31697e-6a2f-48db-9838-a6ca944d3f3b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:17.513117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ea31697e-6a2f-48db-9838-a6ca944d3f3b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:17.713308 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4f63ccb6-2f3c-476a-901b-a49022dc125c tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee" status: 200 len: 345 microversion: 2.1 time: 0.256061 Aug 21 01:25:17.716419 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 13/31] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:25:17 2022] GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee => generated 345 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:17.867513 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2eb2cb54-6c40-435f-bde1-8e4942cdf2ad tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:17.869593 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-2eb2cb54-6c40-435f-bde1-8e4942cdf2ad tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:18.019355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ea31697e-6a2f-48db-9838-a6ca944d3f3b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:18.046316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ea31697e-6a2f-48db-9838-a6ca944d3f3b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1222 microversion: 2.1 time: 0.544628 Aug 21 01:25:18.050485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 19/32] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:17 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1222 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:18.169934 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2eb2cb54-6c40-435f-bde1-8e4942cdf2ad tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:18.180940 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2eb2cb54-6c40-435f-bde1-8e4942cdf2ad tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1215 microversion: 2.1 time: 0.324508 Aug 21 01:25:18.183106 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 14/33] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:17 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1215 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:18.733481 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-37df0c0d-9b13-4cd7-9a64-4daed11ea3fb tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da8187160>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:18.872363 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-37df0c0d-9b13-4cd7-9a64-4daed11ea3fb tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee" status: 200 len: 345 microversion: 2.1 time: 0.142466 Aug 21 01:25:18.874529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 20/34] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:25:18 2022] GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee => generated 345 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:19.094882 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f5c87096-19e2-4e78-99cc-9aa65f03ffd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:19.097994 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f5c87096-19e2-4e78-99cc-9aa65f03ffd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:19.180148 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-16384747-429c-45c6-8743-61e4b15eb505 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Action: 'create', calling method: .version_select at 0x7f9da81c8ca0>, body: {"keypair": {"name": "tempest-keypair-21967027", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEV90aYQnxwvBBfBLry5ajumcaHyWFjIOl0Ypcwxi71fvkETtmzuOoWW3+8VMGhP190XK+XgRhZnAOJKiZoBBRdCCcVIijf8edLimTo/Sgw7DcOGgtV3iljpLMSI7AP4Cg=="}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:19.200281 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-16384747-429c-45c6-8743-61e4b15eb505 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Getting quotas for project 13b534c485db48d8a37e024702f87092. Resources: {'key_pairs'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:25:19.205968 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-16384747-429c-45c6-8743-61e4b15eb505 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Getting quotas for user 0f4a35c46e4741c1b238f24d203b416d and project 13b534c485db48d8a37e024702f87092. Resources: {'key_pairs'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:25:19.318757 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-16384747-429c-45c6-8743-61e4b15eb505 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "POST /compute/v2.1/os-keypairs" status: 201 len: 399 microversion: 2.20 time: 0.431471 Aug 21 01:25:19.318757 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 15/35] 10.222.0.47 () {62 vars in 1200 bytes} [Sun Aug 21 01:25:18 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 434 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Aug 21 01:25:19.335186 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7a55badc-a52c-4cc5-94af-129084582482 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:19.336991 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7a55badc-a52c-4cc5-94af-129084582482 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:19.360865 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7a55badc-a52c-4cc5-94af-129084582482 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:19.361577 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7a55badc-a52c-4cc5-94af-129084582482 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:19.362194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7a55badc-a52c-4cc5-94af-129084582482 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:19.363121 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7a55badc-a52c-4cc5-94af-129084582482 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:19.363923 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7a55badc-a52c-4cc5-94af-129084582482 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:19.364577 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7a55badc-a52c-4cc5-94af-129084582482 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:19.384518 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f5c87096-19e2-4e78-99cc-9aa65f03ffd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:19.437505 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f5c87096-19e2-4e78-99cc-9aa65f03ffd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1222 microversion: 2.1 time: 0.370765 Aug 21 01:25:19.438501 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 21/36] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:19 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1222 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:19.690373 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7a55badc-a52c-4cc5-94af-129084582482 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:19.734172 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7a55badc-a52c-4cc5-94af-129084582482 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1219 microversion: 2.1 time: 0.404704 Aug 21 01:25:19.740779 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 16/37] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:19 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1219 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:19.890411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c8c7c081-fda7-416e-9469-d3b3084db7e6 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da81b2dc0>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:19.994288 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c8c7c081-fda7-416e-9469-d3b3084db7e6 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee" status: 200 len: 345 microversion: 2.1 time: 0.110553 Aug 21 01:25:20.000629 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 22/38] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:25:19 2022] GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee => generated 345 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:20.461753 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b21e1fe6-f165-40c3-a361-c7d1b472e958 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:20.463138 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b21e1fe6-f165-40c3-a361-c7d1b472e958 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:20.470605 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b21e1fe6-f165-40c3-a361-c7d1b472e958 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:20.471533 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b21e1fe6-f165-40c3-a361-c7d1b472e958 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:20.472283 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b21e1fe6-f165-40c3-a361-c7d1b472e958 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:20.473174 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b21e1fe6-f165-40c3-a361-c7d1b472e958 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:20.473821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b21e1fe6-f165-40c3-a361-c7d1b472e958 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:20.474513 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b21e1fe6-f165-40c3-a361-c7d1b472e958 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:20.580708 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b21e1fe6-f165-40c3-a361-c7d1b472e958 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:20.593943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b21e1fe6-f165-40c3-a361-c7d1b472e958 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1226 microversion: 2.1 time: 0.136295 Aug 21 01:25:20.595562 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 17/39] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:20 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1226 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:20.780465 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-5a0a9b6b-5f91-465a-bc56-ef76c595a962 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:20.781400 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-5a0a9b6b-5f91-465a-bc56-ef76c595a962 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:20.793606 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5a0a9b6b-5f91-465a-bc56-ef76c595a962 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:20.794709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5a0a9b6b-5f91-465a-bc56-ef76c595a962 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:20.795828 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5a0a9b6b-5f91-465a-bc56-ef76c595a962 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:20.797488 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5a0a9b6b-5f91-465a-bc56-ef76c595a962 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:20.798360 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5a0a9b6b-5f91-465a-bc56-ef76c595a962 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:20.800386 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5a0a9b6b-5f91-465a-bc56-ef76c595a962 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:21.008604 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2410db63-65ad-482e-8039-0fa86d36c3a8 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da814ca60>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:21.021563 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5a0a9b6b-5f91-465a-bc56-ef76c595a962 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:21.042902 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5a0a9b6b-5f91-465a-bc56-ef76c595a962 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1211 microversion: 2.1 time: 0.271307 Aug 21 01:25:21.052289 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 23/40] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:20 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1211 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:21.102337 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2410db63-65ad-482e-8039-0fa86d36c3a8 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee" status: 200 len: 345 microversion: 2.1 time: 0.098156 Aug 21 01:25:21.103459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 18/41] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:25:21 2022] GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee => generated 345 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:21.169491 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1423642037", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "9ee81273-c1c2-4ebe-9984-21786c936dee"}], "security_groups": [{"name": "tempest-securitygroup--105509774"}], "key_name": "tempest-keypair-1692936143", "user_data": "}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:21.372754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] validate_networks() for [('9ee81273-c1c2-4ebe-9984-21786c936dee', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:25:21.619077 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8bac072d-491a-4537-8835-62f63c251d73 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:21.619704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8bac072d-491a-4537-8835-62f63c251d73 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:21.627781 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8bac072d-491a-4537-8835-62f63c251d73 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:21.628156 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8bac072d-491a-4537-8835-62f63c251d73 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:21.628608 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8bac072d-491a-4537-8835-62f63c251d73 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:21.629321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8bac072d-491a-4537-8835-62f63c251d73 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:21.629775 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8bac072d-491a-4537-8835-62f63c251d73 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:21.630415 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8bac072d-491a-4537-8835-62f63c251d73 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:21.713421 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8bac072d-491a-4537-8835-62f63c251d73 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:21.722554 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8bac072d-491a-4537-8835-62f63c251d73 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1226 microversion: 2.1 time: 0.107738 Aug 21 01:25:21.727412 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 19/42] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:21 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1226 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:22.067600 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-37803780-9258-44e4-9dc8-b6cd37a40a16 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:22.069117 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-37803780-9258-44e4-9dc8-b6cd37a40a16 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:22.082581 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37803780-9258-44e4-9dc8-b6cd37a40a16 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:22.087736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37803780-9258-44e4-9dc8-b6cd37a40a16 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:22.087736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37803780-9258-44e4-9dc8-b6cd37a40a16 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:22.087736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37803780-9258-44e4-9dc8-b6cd37a40a16 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:22.087736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37803780-9258-44e4-9dc8-b6cd37a40a16 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:22.088089 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37803780-9258-44e4-9dc8-b6cd37a40a16 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:22.373207 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-37803780-9258-44e4-9dc8-b6cd37a40a16 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:22.386016 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-37803780-9258-44e4-9dc8-b6cd37a40a16 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1211 microversion: 2.1 time: 0.322842 Aug 21 01:25:22.418829 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 20/43] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:22 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1211 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:22.424839 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-648ddb00-6994-42ec-ae4e-542674fd941a tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da8157280>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:22.519917 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-648ddb00-6994-42ec-ae4e-542674fd941a tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee" status: 200 len: 345 microversion: 2.1 time: 0.099663 Aug 21 01:25:22.520788 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 21/44] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:25:22 2022] GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee => generated 345 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:22.585535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:25:22.587209 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:25:22.587982 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:25:22.588594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:25:22.592028 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:25:22.600031 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:22.600932 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:22.601570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:22.602672 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:22.603368 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:22.604260 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:22.643259 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Getting quotas for project f4a0fcea08b248fe84012875a7243761. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:25:22.651869 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Getting quotas for user 0cd8d822f15943a78575c3744692cdb9 and project f4a0fcea08b248fe84012875a7243761. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:25:22.667770 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:25:22.670030 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:25:22.750085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-39a8a63b-f64c-4122-99d6-35bdb9c54447 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:22.753095 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-39a8a63b-f64c-4122-99d6-35bdb9c54447 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:22.764543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-39a8a63b-f64c-4122-99d6-35bdb9c54447 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:22.766147 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-39a8a63b-f64c-4122-99d6-35bdb9c54447 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:22.767885 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-39a8a63b-f64c-4122-99d6-35bdb9c54447 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:22.769522 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-39a8a63b-f64c-4122-99d6-35bdb9c54447 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:22.771079 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-39a8a63b-f64c-4122-99d6-35bdb9c54447 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:22.771913 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-39a8a63b-f64c-4122-99d6-35bdb9c54447 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:22.831080 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3b7df08b-b4b8-414d-9907-1138e0c3935d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.667393 Aug 21 01:25:22.838886 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 24/45] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:25:21 2022] POST /compute/v2.1/servers => generated 401 bytes in 1677 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:25:22.844661 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-09749d89-83b0-4bb4-8fb7-0ef61eccd589 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:22.848171 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-09749d89-83b0-4bb4-8fb7-0ef61eccd589 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:22.900565 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-39a8a63b-f64c-4122-99d6-35bdb9c54447 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:22.912642 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-39a8a63b-f64c-4122-99d6-35bdb9c54447 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1274 microversion: 2.1 time: 0.166970 Aug 21 01:25:22.915040 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 22/46] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:22 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1274 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:22.949446 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-09749d89-83b0-4bb4-8fb7-0ef61eccd589 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:22.958714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-09749d89-83b0-4bb4-8fb7-0ef61eccd589 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1238 microversion: 2.1 time: 0.118607 Aug 21 01:25:22.959489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 25/47] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:22 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1238 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:23.409747 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b18342c6-f226-4272-a302-d22caf0ab2d3 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:23.410354 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b18342c6-f226-4272-a302-d22caf0ab2d3 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:23.426927 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b18342c6-f226-4272-a302-d22caf0ab2d3 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:23.431186 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b18342c6-f226-4272-a302-d22caf0ab2d3 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:23.431725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b18342c6-f226-4272-a302-d22caf0ab2d3 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:23.432520 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b18342c6-f226-4272-a302-d22caf0ab2d3 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:23.432999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b18342c6-f226-4272-a302-d22caf0ab2d3 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:23.433428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b18342c6-f226-4272-a302-d22caf0ab2d3 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:23.539829 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f97b3e15-0793-44f2-85c0-2f370ae997f3 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da81385e0>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:23.847481 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b18342c6-f226-4272-a302-d22caf0ab2d3 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:23.858223 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b18342c6-f226-4272-a302-d22caf0ab2d3 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1275 microversion: 2.1 time: 0.454844 Aug 21 01:25:23.879954 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 23/48] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:23 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1275 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:23.936406 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f7cd789b-e604-451d-806d-3e9bde926741 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:23.937518 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f7cd789b-e604-451d-806d-3e9bde926741 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:23.949445 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7cd789b-e604-451d-806d-3e9bde926741 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:23.954260 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7cd789b-e604-451d-806d-3e9bde926741 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:23.956446 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7cd789b-e604-451d-806d-3e9bde926741 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:23.961445 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7cd789b-e604-451d-806d-3e9bde926741 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:23.969809 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7cd789b-e604-451d-806d-3e9bde926741 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:23.970411 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7cd789b-e604-451d-806d-3e9bde926741 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:24.003056 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-f97b3e15-0793-44f2-85c0-2f370ae997f3 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] HTTP exception thrown: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f97b3e15-0793-44f2-85c0-2f370ae997f3 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Request method failure captured: Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee HTTP/1.1 Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYlpE0IxnmNgi7UI6prU850cdhWavoXZGs4GneXBCQVthKqGCiyrTKRCpJ1wq7liWPUv1HC3awYWnDyqTyaJxPSRhA1WmuPLZ87DWYRNXNUYK0WVHUTp8zS-BmVwKiS9huGBHXrS3wXfFEkBL4JG9sIM6nXKcoxmgSLa_G4NuuY0yXY Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 1f324851bf32439880d91a81160858ab Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-283509664 Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:25:24.009958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/1f324851bf32439880d91a81160858ab"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_1f324851bf32439880d91a81160858ab"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/1f324851bf32439880d91a81160858ab"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/1f324851bf32439880d91a81160858ab"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-VolumesSnapshotsTestJSON-283509664 Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 1f324851bf32439880d91a81160858ab Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-283509664 Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-VolumesSnapshotsTestJSON-283509664-project Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: dc3e6a6484c74d2cab56e6696404817d Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-VolumesSnapshotsTestJSON-283509664-project Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': 'af225eb8-7cc0-48cb-b0f6-d8d95cff16ee'} Aug 21 01:25:24.011707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. (HTTP 404) (Request-ID: req-dcac36dd-4834-482d-973f-bcf0fc0ad7c4) Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:24.012565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 21 01:25:24.014282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 21 01:25:24.016074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f97b3e15-0793-44f2-85c0-2f370ae997f3 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Returning 404 to user: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f97b3e15-0793-44f2-85c0-2f370ae997f3 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee" status: 404 len: 109 microversion: 2.1 time: 0.482086 Aug 21 01:25:24.018067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 26/49] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:25:23 2022] GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee => generated 109 bytes in 484 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:25:24.048418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1bde61f7-a228-485c-b407-a64e2d320926 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:24.048550 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1bde61f7-a228-485c-b407-a64e2d320926 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:24.252253 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1bde61f7-a228-485c-b407-a64e2d320926 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:24.257900 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f7cd789b-e604-451d-806d-3e9bde926741 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:24.264766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f7cd789b-e604-451d-806d-3e9bde926741 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1282 microversion: 2.1 time: 0.333105 Aug 21 01:25:24.280127 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 24/50] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:23 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1282 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:24.280127 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ef0e3dec-a2a3-47f9-8dd4-d66251c740de tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da80bca60>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:24.280758 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1bde61f7-a228-485c-b407-a64e2d320926 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1238 microversion: 2.1 time: 0.260008 Aug 21 01:25:24.291111 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 27/51] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:24 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1238 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:24.313737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1781473937", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "cb373d64-899b-489f-9b02-6021edfc53f0"}], "security_groups": [{"name": "tempest-securitygroup--2053355159"}], "key_name": "tempest-keypair-21967027", "user_data": "}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:24.346032 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-ef0e3dec-a2a3-47f9-8dd4-d66251c740de tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] HTTP exception thrown: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ef0e3dec-a2a3-47f9-8dd4-d66251c740de tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Request method failure captured: Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: DELETE /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee HTTP/1.1 Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYlpE0IxnmNgi7UI6prU850cdhWavoXZGs4GneXBCQVthKqGCiyrTKRCpJ1wq7liWPUv1HC3awYWnDyqTyaJxPSRhA1WmuPLZ87DWYRNXNUYK0WVHUTp8zS-BmVwKiS9huGBHXrS3wXfFEkBL4JG9sIM6nXKcoxmgSLa_G4NuuY0yXY Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 1f324851bf32439880d91a81160858ab Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-283509664 Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:25:24.351490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/1f324851bf32439880d91a81160858ab"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_1f324851bf32439880d91a81160858ab"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/1f324851bf32439880d91a81160858ab"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/1f324851bf32439880d91a81160858ab"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-VolumesSnapshotsTestJSON-283509664 Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 1f324851bf32439880d91a81160858ab Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-283509664 Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-VolumesSnapshotsTestJSON-283509664-project Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: dc3e6a6484c74d2cab56e6696404817d Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-VolumesSnapshotsTestJSON-283509664-project Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': 'af225eb8-7cc0-48cb-b0f6-d8d95cff16ee'} Aug 21 01:25:24.353341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. (HTTP 404) (Request-ID: req-e5e49ab3-b48a-4ab9-a1b2-e7a60ba6c9fb) Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:24.354232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 21 01:25:24.355358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 21 01:25:24.356736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:25:24.357808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.357808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.357808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:24.357808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.357808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:24.357808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:24.357808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:24.357808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:25:24.357808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:25:24.357808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.357808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.361294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ef0e3dec-a2a3-47f9-8dd4-d66251c740de tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Returning 404 to user: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:25:24.361294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ef0e3dec-a2a3-47f9-8dd4-d66251c740de tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "DELETE /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee" status: 404 len: 109 microversion: 2.1 time: 0.089904 Aug 21 01:25:24.361294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 25/52] 10.222.0.47 () {58 vars in 1248 bytes} [Sun Aug 21 01:25:24 2022] DELETE /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee => generated 109 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:25:24.371413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6c732a7c-76de-4565-97d8-d8bc1a104064 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Calling method '.version_select at 0x7f9da926cee0>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:24.423165 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-6c732a7c-76de-4565-97d8-d8bc1a104064 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] HTTP exception thrown: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6c732a7c-76de-4565-97d8-d8bc1a104064 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Request method failure captured: Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee HTTP/1.1 Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYlpE0IxnmNgi7UI6prU850cdhWavoXZGs4GneXBCQVthKqGCiyrTKRCpJ1wq7liWPUv1HC3awYWnDyqTyaJxPSRhA1WmuPLZ87DWYRNXNUYK0WVHUTp8zS-BmVwKiS9huGBHXrS3wXfFEkBL4JG9sIM6nXKcoxmgSLa_G4NuuY0yXY Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 1f324851bf32439880d91a81160858ab Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-283509664 Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:25:24.425584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/1f324851bf32439880d91a81160858ab"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_1f324851bf32439880d91a81160858ab"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/1f324851bf32439880d91a81160858ab"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/1f324851bf32439880d91a81160858ab"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-VolumesSnapshotsTestJSON-283509664 Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 1f324851bf32439880d91a81160858ab Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-283509664 Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-VolumesSnapshotsTestJSON-283509664-project Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: dc3e6a6484c74d2cab56e6696404817d Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-VolumesSnapshotsTestJSON-283509664-project Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': 'af225eb8-7cc0-48cb-b0f6-d8d95cff16ee'} Aug 21 01:25:24.426977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. (HTTP 404) (Request-ID: req-41725ab5-1091-4a49-aadc-0f025ee1a317) Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:24.427731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 21 01:25:24.430748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 21 01:25:24.431632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6c732a7c-76de-4565-97d8-d8bc1a104064 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] Returning 404 to user: Volume af225eb8-7cc0-48cb-b0f6-d8d95cff16ee could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6c732a7c-76de-4565-97d8-d8bc1a104064 tempest-VolumesSnapshotsTestJSON-283509664 tempest-VolumesSnapshotsTestJSON-283509664-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee" status: 404 len: 109 microversion: 2.1 time: 0.060813 Aug 21 01:25:24.432773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 26/53] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:25:24 2022] GET /compute/v2.1/os-volumes/af225eb8-7cc0-48cb-b0f6-d8d95cff16ee => generated 109 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:25:24.599665 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] validate_networks() for [('cb373d64-899b-489f-9b02-6021edfc53f0', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:25:24.892474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-51cf3aef-5260-46a5-b19f-127561b25914 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:24.894573 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-51cf3aef-5260-46a5-b19f-127561b25914 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:24.902482 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51cf3aef-5260-46a5-b19f-127561b25914 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:24.903362 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51cf3aef-5260-46a5-b19f-127561b25914 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:24.904075 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51cf3aef-5260-46a5-b19f-127561b25914 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:24.906372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51cf3aef-5260-46a5-b19f-127561b25914 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:24.907120 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51cf3aef-5260-46a5-b19f-127561b25914 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:24.907982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51cf3aef-5260-46a5-b19f-127561b25914 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:25.073844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-51cf3aef-5260-46a5-b19f-127561b25914 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:25.093426 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-51cf3aef-5260-46a5-b19f-127561b25914 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1273 microversion: 2.1 time: 0.208670 Aug 21 01:25:25.094386 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 27/54] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:24 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1273 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:25.297244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-660186b6-ae4d-4165-818d-f7af7dcf0ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:25.299490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-660186b6-ae4d-4165-818d-f7af7dcf0ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:25.313806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-660186b6-ae4d-4165-818d-f7af7dcf0ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:25.314516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-660186b6-ae4d-4165-818d-f7af7dcf0ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:25.318351 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-660186b6-ae4d-4165-818d-f7af7dcf0ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:25.319303 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-660186b6-ae4d-4165-818d-f7af7dcf0ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:25.322531 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-660186b6-ae4d-4165-818d-f7af7dcf0ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:25.323245 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-660186b6-ae4d-4165-818d-f7af7dcf0ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:25.332539 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:25:25.559437 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-660186b6-ae4d-4165-818d-f7af7dcf0ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:25.591687 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-660186b6-ae4d-4165-818d-f7af7dcf0ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1280 microversion: 2.1 time: 0.291893 Aug 21 01:25:25.592554 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 28/55] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:25 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1280 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:25.615216 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-37f2af57-4a5d-49b1-a62a-6b7b77c6b28b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:25.620868 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-37f2af57-4a5d-49b1-a62a-6b7b77c6b28b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:25.629598 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37f2af57-4a5d-49b1-a62a-6b7b77c6b28b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:25.630406 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37f2af57-4a5d-49b1-a62a-6b7b77c6b28b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:25.631067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37f2af57-4a5d-49b1-a62a-6b7b77c6b28b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:25.632205 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37f2af57-4a5d-49b1-a62a-6b7b77c6b28b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:25.633198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37f2af57-4a5d-49b1-a62a-6b7b77c6b28b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:25.633843 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37f2af57-4a5d-49b1-a62a-6b7b77c6b28b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:25.865085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-37f2af57-4a5d-49b1-a62a-6b7b77c6b28b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:25.877702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-37f2af57-4a5d-49b1-a62a-6b7b77c6b28b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1242 microversion: 2.1 time: 0.271814 Aug 21 01:25:25.878426 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 29/56] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:25 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1242 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:26.085773 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:25:26.086272 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:25:26.086444 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:25:26.087034 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:25:26.087586 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:25:26.096829 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:26.097553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:26.098148 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:26.099076 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:26.099785 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:26.101000 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:26.116559 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-50c13f62-d4c5-4d8e-889f-357f9d74640c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:26.124825 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-50c13f62-d4c5-4d8e-889f-357f9d74640c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:26.129024 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Getting quotas for project 13b534c485db48d8a37e024702f87092. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:25:26.134154 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Getting quotas for user 0f4a35c46e4741c1b238f24d203b416d and project 13b534c485db48d8a37e024702f87092. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:25:26.142449 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50c13f62-d4c5-4d8e-889f-357f9d74640c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:26.142959 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50c13f62-d4c5-4d8e-889f-357f9d74640c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:26.143416 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50c13f62-d4c5-4d8e-889f-357f9d74640c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:26.144152 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50c13f62-d4c5-4d8e-889f-357f9d74640c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:26.144618 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50c13f62-d4c5-4d8e-889f-357f9d74640c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:26.145052 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50c13f62-d4c5-4d8e-889f-357f9d74640c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:26.146680 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:25:26.149856 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:25:26.384706 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e2e39438-8ea3-4451-8e3e-ffab4b975c88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 2.087238 Aug 21 01:25:26.392864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 28/57] 10.222.0.47 () {62 vars in 1188 bytes} [Sun Aug 21 01:25:24 2022] POST /compute/v2.1/servers => generated 402 bytes in 2096 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Aug 21 01:25:26.405515 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-abc77256-0abf-48e0-8c7a-b119c382609d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:26.406437 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-abc77256-0abf-48e0-8c7a-b119c382609d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:26.463635 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-50c13f62-d4c5-4d8e-889f-357f9d74640c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:26.474423 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-50c13f62-d4c5-4d8e-889f-357f9d74640c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.361692 Aug 21 01:25:26.475543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 30/58] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:26 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:26.554447 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-abc77256-0abf-48e0-8c7a-b119c382609d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:26.571197 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-abc77256-0abf-48e0-8c7a-b119c382609d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:26.572848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-abc77256-0abf-48e0-8c7a-b119c382609d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:26.573988 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-abc77256-0abf-48e0-8c7a-b119c382609d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1280 microversion: 2.20 time: 0.177220 Aug 21 01:25:26.574536 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 29/59] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:26 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1280 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:25:26.616486 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ede81e83-eb70-459c-9373-de3fcdf1e1e8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:26.617092 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-ede81e83-eb70-459c-9373-de3fcdf1e1e8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:26.632280 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ede81e83-eb70-459c-9373-de3fcdf1e1e8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:26.632280 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ede81e83-eb70-459c-9373-de3fcdf1e1e8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:26.632280 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ede81e83-eb70-459c-9373-de3fcdf1e1e8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:26.632280 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ede81e83-eb70-459c-9373-de3fcdf1e1e8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:26.633463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ede81e83-eb70-459c-9373-de3fcdf1e1e8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:26.634066 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ede81e83-eb70-459c-9373-de3fcdf1e1e8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:26.903179 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-468621d7-892e-455f-9051-90572655f34f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:26.906377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-468621d7-892e-455f-9051-90572655f34f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:26.939872 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-468621d7-892e-455f-9051-90572655f34f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:26.945532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-468621d7-892e-455f-9051-90572655f34f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:26.946126 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-468621d7-892e-455f-9051-90572655f34f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:26.946818 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-468621d7-892e-455f-9051-90572655f34f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:26.947288 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-468621d7-892e-455f-9051-90572655f34f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:26.947721 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-468621d7-892e-455f-9051-90572655f34f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:27.008234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ede81e83-eb70-459c-9373-de3fcdf1e1e8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:27.021772 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ede81e83-eb70-459c-9373-de3fcdf1e1e8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1280 microversion: 2.1 time: 0.410153 Aug 21 01:25:27.024439 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 31/60] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:26 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1280 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:27.088308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-468621d7-892e-455f-9051-90572655f34f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:27.096609 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-468621d7-892e-455f-9051-90572655f34f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1242 microversion: 2.1 time: 0.197354 Aug 21 01:25:27.097282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 30/61] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:26 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1242 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:27.496613 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-cb3e3309-7136-4e31-8261-15c40bb4279f tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:27.497245 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-cb3e3309-7136-4e31-8261-15c40bb4279f tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:27.504557 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb3e3309-7136-4e31-8261-15c40bb4279f tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:27.505175 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb3e3309-7136-4e31-8261-15c40bb4279f tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:27.505626 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb3e3309-7136-4e31-8261-15c40bb4279f tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:27.506911 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb3e3309-7136-4e31-8261-15c40bb4279f tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:27.507380 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb3e3309-7136-4e31-8261-15c40bb4279f tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:27.507814 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb3e3309-7136-4e31-8261-15c40bb4279f tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:27.606584 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9851c40f-a427-48fd-8d15-237b2f9402cd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:27.607479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9851c40f-a427-48fd-8d15-237b2f9402cd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:27.646453 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=88,nova_api:INSERT=12 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:25:28.004073 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9851c40f-a427-48fd-8d15-237b2f9402cd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:28.020078 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9851c40f-a427-48fd-8d15-237b2f9402cd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:28.026343 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9851c40f-a427-48fd-8d15-237b2f9402cd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:28.032012 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9851c40f-a427-48fd-8d15-237b2f9402cd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1280 microversion: 2.20 time: 0.426661 Aug 21 01:25:28.049121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 31/62] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:27 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1280 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:28.066712 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-519ab4e0-9146-45dd-97f4-f770ac3b55c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:28.066712 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-519ab4e0-9146-45dd-97f4-f770ac3b55c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:28.079988 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-519ab4e0-9146-45dd-97f4-f770ac3b55c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:28.083347 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-519ab4e0-9146-45dd-97f4-f770ac3b55c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:28.086273 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-519ab4e0-9146-45dd-97f4-f770ac3b55c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:28.092506 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-519ab4e0-9146-45dd-97f4-f770ac3b55c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:28.101107 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-519ab4e0-9146-45dd-97f4-f770ac3b55c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:28.105249 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-519ab4e0-9146-45dd-97f4-f770ac3b55c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:28.110696 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-cb3e3309-7136-4e31-8261-15c40bb4279f tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:28.123293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-cb3e3309-7136-4e31-8261-15c40bb4279f tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.628411 Aug 21 01:25:28.123293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 32/63] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:27 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:28.126505 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8166d432-70e1-4f57-90b1-e41406a2c7fe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:28.127449 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8166d432-70e1-4f57-90b1-e41406a2c7fe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:28.143293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8166d432-70e1-4f57-90b1-e41406a2c7fe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:28.145033 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8166d432-70e1-4f57-90b1-e41406a2c7fe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:28.147162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8166d432-70e1-4f57-90b1-e41406a2c7fe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:28.149535 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8166d432-70e1-4f57-90b1-e41406a2c7fe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:28.151733 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8166d432-70e1-4f57-90b1-e41406a2c7fe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:28.152661 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8166d432-70e1-4f57-90b1-e41406a2c7fe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:28.325017 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8166d432-70e1-4f57-90b1-e41406a2c7fe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:28.353797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8166d432-70e1-4f57-90b1-e41406a2c7fe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1242 microversion: 2.1 time: 0.213035 Aug 21 01:25:28.356224 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 33/64] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:28 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1242 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:28.465207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-519ab4e0-9146-45dd-97f4-f770ac3b55c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:28.475581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-519ab4e0-9146-45dd-97f4-f770ac3b55c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.425260 Aug 21 01:25:28.499898 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 32/65] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:28 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:29.070368 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-63c6972b-e9b7-4f4c-869c-de830ba0a11a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:29.075824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-63c6972b-e9b7-4f4c-869c-de830ba0a11a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:29.101840 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63c6972b-e9b7-4f4c-869c-de830ba0a11a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:29.117194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63c6972b-e9b7-4f4c-869c-de830ba0a11a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:29.118142 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63c6972b-e9b7-4f4c-869c-de830ba0a11a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:29.121627 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63c6972b-e9b7-4f4c-869c-de830ba0a11a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:29.126173 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63c6972b-e9b7-4f4c-869c-de830ba0a11a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:29.131851 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63c6972b-e9b7-4f4c-869c-de830ba0a11a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:29.145308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d502c7c1-53c5-45b6-b54e-8669d36a5537 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:29.146528 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d502c7c1-53c5-45b6-b54e-8669d36a5537 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:29.162355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d502c7c1-53c5-45b6-b54e-8669d36a5537 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:29.174096 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d502c7c1-53c5-45b6-b54e-8669d36a5537 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:29.176245 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d502c7c1-53c5-45b6-b54e-8669d36a5537 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:29.185709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d502c7c1-53c5-45b6-b54e-8669d36a5537 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:29.187590 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d502c7c1-53c5-45b6-b54e-8669d36a5537 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:29.188586 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d502c7c1-53c5-45b6-b54e-8669d36a5537 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:29.213684 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=18 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:25:30.057462 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-63c6972b-e9b7-4f4c-869c-de830ba0a11a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:30.067025 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-63c6972b-e9b7-4f4c-869c-de830ba0a11a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:30.076130 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-63c6972b-e9b7-4f4c-869c-de830ba0a11a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:30.077388 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-63c6972b-e9b7-4f4c-869c-de830ba0a11a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1284 microversion: 2.20 time: 1.016674 Aug 21 01:25:30.079366 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 34/66] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:29 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1284 bytes in 1020 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:30.084048 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-97ad97de-78eb-4424-83e5-d14c637ab9a3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:30.084723 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-97ad97de-78eb-4424-83e5-d14c637ab9a3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:30.117681 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-97ad97de-78eb-4424-83e5-d14c637ab9a3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:30.118183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-97ad97de-78eb-4424-83e5-d14c637ab9a3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:30.124095 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-97ad97de-78eb-4424-83e5-d14c637ab9a3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:30.125102 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-97ad97de-78eb-4424-83e5-d14c637ab9a3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:30.125720 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-97ad97de-78eb-4424-83e5-d14c637ab9a3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:30.126294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-97ad97de-78eb-4424-83e5-d14c637ab9a3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:30.310928 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-97ad97de-78eb-4424-83e5-d14c637ab9a3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:30.323757 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-97ad97de-78eb-4424-83e5-d14c637ab9a3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1290 microversion: 2.1 time: 0.243243 Aug 21 01:25:30.326257 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 35/67] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:30 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1290 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:30.337996 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5723db70-efca-440a-bf99-d262e66d9713 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:30.356201 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5723db70-efca-440a-bf99-d262e66d9713 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:30.364306 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d502c7c1-53c5-45b6-b54e-8669d36a5537 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:30.368179 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5723db70-efca-440a-bf99-d262e66d9713 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:30.368962 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5723db70-efca-440a-bf99-d262e66d9713 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:30.369590 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5723db70-efca-440a-bf99-d262e66d9713 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:30.375407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5723db70-efca-440a-bf99-d262e66d9713 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:30.375891 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5723db70-efca-440a-bf99-d262e66d9713 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:30.376378 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5723db70-efca-440a-bf99-d262e66d9713 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:30.383245 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d502c7c1-53c5-45b6-b54e-8669d36a5537 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 1.242850 Aug 21 01:25:30.384634 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 33/68] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:29 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 1245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:30.386376 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=100 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:25:31.109336 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6b622744-af11-430b-98a3-5ac86cb9d121 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:31.109336 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6b622744-af11-430b-98a3-5ac86cb9d121 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:31.119100 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6b622744-af11-430b-98a3-5ac86cb9d121 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:31.121261 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6b622744-af11-430b-98a3-5ac86cb9d121 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:31.121970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6b622744-af11-430b-98a3-5ac86cb9d121 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:31.125265 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6b622744-af11-430b-98a3-5ac86cb9d121 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:31.127270 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6b622744-af11-430b-98a3-5ac86cb9d121 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:31.127935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6b622744-af11-430b-98a3-5ac86cb9d121 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:31.207085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5723db70-efca-440a-bf99-d262e66d9713 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:31.217148 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5723db70-efca-440a-bf99-d262e66d9713 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.888240 Aug 21 01:25:31.217752 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 36/69] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:30 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 890 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:31.354228 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5b6edc2e-70ee-4879-9279-e5f01aebd77e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:31.354797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5b6edc2e-70ee-4879-9279-e5f01aebd77e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:31.362387 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5b6edc2e-70ee-4879-9279-e5f01aebd77e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:31.367115 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5b6edc2e-70ee-4879-9279-e5f01aebd77e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:31.367780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5b6edc2e-70ee-4879-9279-e5f01aebd77e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:31.368739 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5b6edc2e-70ee-4879-9279-e5f01aebd77e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:31.369374 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5b6edc2e-70ee-4879-9279-e5f01aebd77e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:31.370239 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5b6edc2e-70ee-4879-9279-e5f01aebd77e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:31.384421 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6b622744-af11-430b-98a3-5ac86cb9d121 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:31.394785 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6b622744-af11-430b-98a3-5ac86cb9d121 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:31.402828 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6b622744-af11-430b-98a3-5ac86cb9d121 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:31.405020 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6b622744-af11-430b-98a3-5ac86cb9d121 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1332 microversion: 2.20 time: 0.304433 Aug 21 01:25:31.436261 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 34/70] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:31 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1332 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:31.468638 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-98ba5be4-8333-4cfe-8952-6f200bcefe9b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:31.472233 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-98ba5be4-8333-4cfe-8952-6f200bcefe9b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:31.487577 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-98ba5be4-8333-4cfe-8952-6f200bcefe9b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:31.488536 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-98ba5be4-8333-4cfe-8952-6f200bcefe9b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:31.494228 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-98ba5be4-8333-4cfe-8952-6f200bcefe9b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:31.494228 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-98ba5be4-8333-4cfe-8952-6f200bcefe9b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:31.495207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-98ba5be4-8333-4cfe-8952-6f200bcefe9b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:31.495964 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-98ba5be4-8333-4cfe-8952-6f200bcefe9b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:31.599187 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5b6edc2e-70ee-4879-9279-e5f01aebd77e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:31.617240 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5b6edc2e-70ee-4879-9279-e5f01aebd77e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1298 microversion: 2.1 time: 0.270877 Aug 21 01:25:31.648264 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 37/71] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:31 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1298 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:32.249851 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-98ba5be4-8333-4cfe-8952-6f200bcefe9b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:32.265017 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-98ba5be4-8333-4cfe-8952-6f200bcefe9b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.826746 Aug 21 01:25:32.265622 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 35/72] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:31 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 828 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:32.283578 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a9dda98e-c2fc-44d2-8e9a-30651929234e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:32.287473 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a9dda98e-c2fc-44d2-8e9a-30651929234e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:32.304523 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a9dda98e-c2fc-44d2-8e9a-30651929234e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:32.305280 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a9dda98e-c2fc-44d2-8e9a-30651929234e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:32.305968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a9dda98e-c2fc-44d2-8e9a-30651929234e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:32.306841 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a9dda98e-c2fc-44d2-8e9a-30651929234e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:32.307443 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a9dda98e-c2fc-44d2-8e9a-30651929234e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:32.308017 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a9dda98e-c2fc-44d2-8e9a-30651929234e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:32.376145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-806dcd65-870d-42d7-8d47-34e1718b8d0f req-71c04b0c-2840-46e9-83eb-c5a3295b73c6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "90e88110-1175-4ccf-bf6b-8efb43ab945e", "tag": "f56acc10-3115-43fe-929a-cb8d30842aef"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:32.403911 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-806dcd65-870d-42d7-8d47-34e1718b8d0f req-71c04b0c-2840-46e9-83eb-c5a3295b73c6 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:32.404458 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-806dcd65-870d-42d7-8d47-34e1718b8d0f req-71c04b0c-2840-46e9-83eb-c5a3295b73c6 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:32.404891 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-806dcd65-870d-42d7-8d47-34e1718b8d0f req-71c04b0c-2840-46e9-83eb-c5a3295b73c6 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:32.440935 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-806dcd65-870d-42d7-8d47-34e1718b8d0f req-71c04b0c-2840-46e9-83eb-c5a3295b73c6 service nova] Creating event network-changed:f56acc10-3115-43fe-929a-cb8d30842aef for instance 90e88110-1175-4ccf-bf6b-8efb43ab945e on hv-cinder-maste Aug 21 01:25:32.441368 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-806dcd65-870d-42d7-8d47-34e1718b8d0f req-71c04b0c-2840-46e9-83eb-c5a3295b73c6 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=121277) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Aug 21 01:25:32.441757 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-806dcd65-870d-42d7-8d47-34e1718b8d0f req-71c04b0c-2840-46e9-83eb-c5a3295b73c6 service nova] Acquired lock "compute-rpcapi-router" {{(pid=121277) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Aug 21 01:25:32.443013 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ec86720f-4cd2-4b6b-8e64-084dda4eec88 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:32.443821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ec86720f-4cd2-4b6b-8e64-084dda4eec88 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:32.444477 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ec86720f-4cd2-4b6b-8e64-084dda4eec88 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:32.445372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ec86720f-4cd2-4b6b-8e64-084dda4eec88 None None] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:32.445976 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ec86720f-4cd2-4b6b-8e64-084dda4eec88 None None] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:32.446554 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ec86720f-4cd2-4b6b-8e64-084dda4eec88 None None] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:32.456510 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.compute.rpcapi [None req-ec86720f-4cd2-4b6b-8e64-084dda4eec88 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Aug 21 01:25:32.457655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ec86720f-4cd2-4b6b-8e64-084dda4eec88 None None] Releasing lock "compute-rpcapi-router" {{(pid=121277) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Aug 21 01:25:32.476943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ec86720f-4cd2-4b6b-8e64-084dda4eec88 None None] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.808183 Aug 21 01:25:32.477816 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 38/73] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:25:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 810 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:32.483107 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-525cd1c6-e2f4-4410-b10a-6384ef524c49 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:32.485515 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-525cd1c6-e2f4-4410-b10a-6384ef524c49 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:32.493885 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-525cd1c6-e2f4-4410-b10a-6384ef524c49 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:32.500539 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-525cd1c6-e2f4-4410-b10a-6384ef524c49 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:32.500539 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-525cd1c6-e2f4-4410-b10a-6384ef524c49 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:32.500539 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-525cd1c6-e2f4-4410-b10a-6384ef524c49 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:32.500539 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-525cd1c6-e2f4-4410-b10a-6384ef524c49 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:32.500925 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-525cd1c6-e2f4-4410-b10a-6384ef524c49 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:32.852829 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-525cd1c6-e2f4-4410-b10a-6384ef524c49 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:32.862607 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-525cd1c6-e2f4-4410-b10a-6384ef524c49 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:32.864328 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-525cd1c6-e2f4-4410-b10a-6384ef524c49 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:32.866124 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-525cd1c6-e2f4-4410-b10a-6384ef524c49 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1340 microversion: 2.20 time: 0.387089 Aug 21 01:25:32.867528 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 39/74] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:32 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1340 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:32.872594 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6cbc54ca-f7e7-44f0-a8af-5fecbac5b7fb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:32.873442 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6cbc54ca-f7e7-44f0-a8af-5fecbac5b7fb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:32.879821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6cbc54ca-f7e7-44f0-a8af-5fecbac5b7fb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:32.880531 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6cbc54ca-f7e7-44f0-a8af-5fecbac5b7fb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:32.881189 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6cbc54ca-f7e7-44f0-a8af-5fecbac5b7fb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:32.882234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6cbc54ca-f7e7-44f0-a8af-5fecbac5b7fb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:32.888219 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6cbc54ca-f7e7-44f0-a8af-5fecbac5b7fb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:32.889037 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6cbc54ca-f7e7-44f0-a8af-5fecbac5b7fb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:32.897764 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a9dda98e-c2fc-44d2-8e9a-30651929234e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:32.921811 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a9dda98e-c2fc-44d2-8e9a-30651929234e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.654374 Aug 21 01:25:32.938403 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 36/75] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:32 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 671 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:33.030835 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6cbc54ca-f7e7-44f0-a8af-5fecbac5b7fb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:33.042998 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6cbc54ca-f7e7-44f0-a8af-5fecbac5b7fb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1296 microversion: 2.1 time: 0.174159 Aug 21 01:25:33.058599 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 40/76] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:32 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1296 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:33.182721 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-2bb61e9a-7f18-47e0-96e3-0707a375ca65 req-a6b2149b-7855-48f2-b958-45f5ea37c591 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "031af209-ae76-47b8-8481-88ec4053affa", "tag": "667f35ba-e131-4619-82ff-965a65cd85fd"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:33.197154 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2bb61e9a-7f18-47e0-96e3-0707a375ca65 req-a6b2149b-7855-48f2-b958-45f5ea37c591 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:33.197955 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2bb61e9a-7f18-47e0-96e3-0707a375ca65 req-a6b2149b-7855-48f2-b958-45f5ea37c591 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:33.198470 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2bb61e9a-7f18-47e0-96e3-0707a375ca65 req-a6b2149b-7855-48f2-b958-45f5ea37c591 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:33.221948 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-2bb61e9a-7f18-47e0-96e3-0707a375ca65 req-a6b2149b-7855-48f2-b958-45f5ea37c591 service nova] Creating event network-changed:667f35ba-e131-4619-82ff-965a65cd85fd for instance 031af209-ae76-47b8-8481-88ec4053affa on hv-cinder-maste Aug 21 01:25:33.222663 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2bb61e9a-7f18-47e0-96e3-0707a375ca65 req-a6b2149b-7855-48f2-b958-45f5ea37c591 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=121276) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Aug 21 01:25:33.223058 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2bb61e9a-7f18-47e0-96e3-0707a375ca65 req-a6b2149b-7855-48f2-b958-45f5ea37c591 service nova] Acquired lock "compute-rpcapi-router" {{(pid=121276) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Aug 21 01:25:33.224126 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8347c82-af77-4c11-9a05-11c56cabbb71 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:33.226028 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8347c82-af77-4c11-9a05-11c56cabbb71 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:33.226855 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8347c82-af77-4c11-9a05-11c56cabbb71 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:33.227788 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8347c82-af77-4c11-9a05-11c56cabbb71 None None] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:33.228071 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8347c82-af77-4c11-9a05-11c56cabbb71 None None] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:33.228541 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8347c82-af77-4c11-9a05-11c56cabbb71 None None] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:33.239448 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.compute.rpcapi [None req-c8347c82-af77-4c11-9a05-11c56cabbb71 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Aug 21 01:25:33.240695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8347c82-af77-4c11-9a05-11c56cabbb71 None None] Releasing lock "compute-rpcapi-router" {{(pid=121276) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Aug 21 01:25:33.262237 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c8347c82-af77-4c11-9a05-11c56cabbb71 None None] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.318232 Aug 21 01:25:33.262682 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 37/77] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:25:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:33.291695 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5901c25f-82ac-4cdb-af70-0a2f493e1879 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:33.296603 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5901c25f-82ac-4cdb-af70-0a2f493e1879 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:33.300174 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5901c25f-82ac-4cdb-af70-0a2f493e1879 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:33.300744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5901c25f-82ac-4cdb-af70-0a2f493e1879 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:33.301250 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5901c25f-82ac-4cdb-af70-0a2f493e1879 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:33.302022 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5901c25f-82ac-4cdb-af70-0a2f493e1879 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:33.302540 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5901c25f-82ac-4cdb-af70-0a2f493e1879 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:33.302984 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5901c25f-82ac-4cdb-af70-0a2f493e1879 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:33.533182 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5901c25f-82ac-4cdb-af70-0a2f493e1879 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:33.556429 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5901c25f-82ac-4cdb-af70-0a2f493e1879 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.270021 Aug 21 01:25:33.566881 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 41/78] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:33 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:33.893947 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3fbac416-ee19-4437-9b85-3baa9ef08382 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:33.894724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3fbac416-ee19-4437-9b85-3baa9ef08382 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:33.922578 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fbac416-ee19-4437-9b85-3baa9ef08382 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:33.922578 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fbac416-ee19-4437-9b85-3baa9ef08382 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:33.922578 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fbac416-ee19-4437-9b85-3baa9ef08382 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:33.922578 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fbac416-ee19-4437-9b85-3baa9ef08382 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:33.923319 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fbac416-ee19-4437-9b85-3baa9ef08382 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:33.923319 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fbac416-ee19-4437-9b85-3baa9ef08382 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:33.949196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b3515ba0-bdba-4655-a04a-35f9bcf49291 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:33.954171 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b3515ba0-bdba-4655-a04a-35f9bcf49291 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:33.961459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b3515ba0-bdba-4655-a04a-35f9bcf49291 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:33.961884 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b3515ba0-bdba-4655-a04a-35f9bcf49291 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:33.962335 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b3515ba0-bdba-4655-a04a-35f9bcf49291 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:33.963076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b3515ba0-bdba-4655-a04a-35f9bcf49291 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:33.963532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b3515ba0-bdba-4655-a04a-35f9bcf49291 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:33.963976 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b3515ba0-bdba-4655-a04a-35f9bcf49291 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:33.982142 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3fbac416-ee19-4437-9b85-3baa9ef08382 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:33.991663 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3fbac416-ee19-4437-9b85-3baa9ef08382 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:33.993467 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3fbac416-ee19-4437-9b85-3baa9ef08382 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:33.994634 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3fbac416-ee19-4437-9b85-3baa9ef08382 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1340 microversion: 2.20 time: 0.105488 Aug 21 01:25:33.995136 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 38/79] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:33 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1340 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:34.068361 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ba3fb132-0a91-46c9-9417-d872aa773714 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:34.071304 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ba3fb132-0a91-46c9-9417-d872aa773714 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:34.076341 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ba3fb132-0a91-46c9-9417-d872aa773714 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:34.076838 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ba3fb132-0a91-46c9-9417-d872aa773714 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:34.077275 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ba3fb132-0a91-46c9-9417-d872aa773714 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:34.077980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ba3fb132-0a91-46c9-9417-d872aa773714 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:34.078502 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ba3fb132-0a91-46c9-9417-d872aa773714 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:34.078970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ba3fb132-0a91-46c9-9417-d872aa773714 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:34.103210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b3515ba0-bdba-4655-a04a-35f9bcf49291 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:34.111404 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b3515ba0-bdba-4655-a04a-35f9bcf49291 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.166020 Aug 21 01:25:34.118597 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 42/80] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:33 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:34.142644 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ba3fb132-0a91-46c9-9417-d872aa773714 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:34.150321 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ba3fb132-0a91-46c9-9417-d872aa773714 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1296 microversion: 2.1 time: 0.089099 Aug 21 01:25:34.151716 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 39/81] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:34 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1296 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:34.576584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3a9fb175-f3b2-4a00-bf73-73736c4ffecc tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:34.577199 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3a9fb175-f3b2-4a00-bf73-73736c4ffecc tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:34.583541 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3a9fb175-f3b2-4a00-bf73-73736c4ffecc tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:34.584872 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3a9fb175-f3b2-4a00-bf73-73736c4ffecc tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:34.584997 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3a9fb175-f3b2-4a00-bf73-73736c4ffecc tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:34.585720 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3a9fb175-f3b2-4a00-bf73-73736c4ffecc tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:34.586097 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3a9fb175-f3b2-4a00-bf73-73736c4ffecc tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:34.588395 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3a9fb175-f3b2-4a00-bf73-73736c4ffecc tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:34.717869 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3a9fb175-f3b2-4a00-bf73-73736c4ffecc tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:34.725653 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3a9fb175-f3b2-4a00-bf73-73736c4ffecc tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.152924 Aug 21 01:25:34.726278 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 43/82] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:34 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:35.017974 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2f15afef-656d-4062-8526-dcc59cec7c25 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:35.019261 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2f15afef-656d-4062-8526-dcc59cec7c25 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:35.028964 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2f15afef-656d-4062-8526-dcc59cec7c25 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:35.029790 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2f15afef-656d-4062-8526-dcc59cec7c25 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:35.036880 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2f15afef-656d-4062-8526-dcc59cec7c25 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:35.036880 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2f15afef-656d-4062-8526-dcc59cec7c25 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:35.036880 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2f15afef-656d-4062-8526-dcc59cec7c25 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:35.036880 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2f15afef-656d-4062-8526-dcc59cec7c25 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:35.133917 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a50842f1-c2c2-4c10-a8dd-f4d49542d4b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:35.134660 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a50842f1-c2c2-4c10-a8dd-f4d49542d4b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:35.144031 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a50842f1-c2c2-4c10-a8dd-f4d49542d4b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:35.163220 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a50842f1-c2c2-4c10-a8dd-f4d49542d4b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:35.163220 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a50842f1-c2c2-4c10-a8dd-f4d49542d4b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:35.163220 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a50842f1-c2c2-4c10-a8dd-f4d49542d4b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:35.163220 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a50842f1-c2c2-4c10-a8dd-f4d49542d4b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:35.165612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a50842f1-c2c2-4c10-a8dd-f4d49542d4b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:35.179463 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2f15afef-656d-4062-8526-dcc59cec7c25 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:35.195485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2f15afef-656d-4062-8526-dcc59cec7c25 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:35.203299 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2f15afef-656d-4062-8526-dcc59cec7c25 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:35.203299 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2f15afef-656d-4062-8526-dcc59cec7c25 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1340 microversion: 2.20 time: 0.185357 Aug 21 01:25:35.217270 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 40/83] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:35 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1340 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:25:35.240793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9ba517b9-554e-4540-8f24-6f7213eeb2e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:35.241751 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9ba517b9-554e-4540-8f24-6f7213eeb2e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:35.248330 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ba517b9-554e-4540-8f24-6f7213eeb2e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:35.249077 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ba517b9-554e-4540-8f24-6f7213eeb2e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:35.249680 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ba517b9-554e-4540-8f24-6f7213eeb2e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:35.250547 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ba517b9-554e-4540-8f24-6f7213eeb2e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:35.251152 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ba517b9-554e-4540-8f24-6f7213eeb2e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:35.251731 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ba517b9-554e-4540-8f24-6f7213eeb2e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:35.419803 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9ba517b9-554e-4540-8f24-6f7213eeb2e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:35.442633 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9ba517b9-554e-4540-8f24-6f7213eeb2e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1296 microversion: 2.1 time: 0.223896 Aug 21 01:25:35.448055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 41/84] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:35 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1296 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:35.454639 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-2f79f890-7ce0-4eed-82a2-b8a271129469 req-40882bd3-004a-4386-bc23-9d15dad308cd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "80c72a3e-79b5-48fa-9c5f-26a858b589de", "tag": "f7aa7447-c670-4701-a6ac-18f99415ce92"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:35.476697 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2f79f890-7ce0-4eed-82a2-b8a271129469 req-40882bd3-004a-4386-bc23-9d15dad308cd service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:35.479604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2f79f890-7ce0-4eed-82a2-b8a271129469 req-40882bd3-004a-4386-bc23-9d15dad308cd service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:35.480319 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2f79f890-7ce0-4eed-82a2-b8a271129469 req-40882bd3-004a-4386-bc23-9d15dad308cd service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:35.511176 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-2f79f890-7ce0-4eed-82a2-b8a271129469 req-40882bd3-004a-4386-bc23-9d15dad308cd service nova] Creating event network-changed:f7aa7447-c670-4701-a6ac-18f99415ce92 for instance 80c72a3e-79b5-48fa-9c5f-26a858b589de on hv-cinder-maste Aug 21 01:25:35.517819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-2f79f890-7ce0-4eed-82a2-b8a271129469 req-40882bd3-004a-4386-bc23-9d15dad308cd service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068184 Aug 21 01:25:35.519985 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 42/85] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:25:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:35.547901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a50842f1-c2c2-4c10-a8dd-f4d49542d4b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:35.557151 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a50842f1-c2c2-4c10-a8dd-f4d49542d4b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.427861 Aug 21 01:25:35.571442 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 44/86] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:35 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:35.747967 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fc555c19-bc0e-40b1-92a9-212944e78673 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:35.750834 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fc555c19-bc0e-40b1-92a9-212944e78673 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:35.760612 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fc555c19-bc0e-40b1-92a9-212944e78673 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:35.761384 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fc555c19-bc0e-40b1-92a9-212944e78673 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:35.761924 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fc555c19-bc0e-40b1-92a9-212944e78673 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:35.762743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fc555c19-bc0e-40b1-92a9-212944e78673 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:35.763373 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fc555c19-bc0e-40b1-92a9-212944e78673 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:35.763945 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fc555c19-bc0e-40b1-92a9-212944e78673 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:36.108458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fc555c19-bc0e-40b1-92a9-212944e78673 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:36.108458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fc555c19-bc0e-40b1-92a9-212944e78673 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.312832 Aug 21 01:25:36.108458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 43/87] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:35 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:36.242311 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-17486961-43b5-427c-afbc-51edccddb3c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:36.247832 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-17486961-43b5-427c-afbc-51edccddb3c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:36.259404 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:25:36.265223 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-17486961-43b5-427c-afbc-51edccddb3c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:36.266067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-17486961-43b5-427c-afbc-51edccddb3c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:36.267219 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-17486961-43b5-427c-afbc-51edccddb3c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:36.268044 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-17486961-43b5-427c-afbc-51edccddb3c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:36.268633 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-17486961-43b5-427c-afbc-51edccddb3c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:36.269167 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-17486961-43b5-427c-afbc-51edccddb3c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:36.369968 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-17486961-43b5-427c-afbc-51edccddb3c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:36.378926 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-17486961-43b5-427c-afbc-51edccddb3c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:36.380553 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-17486961-43b5-427c-afbc-51edccddb3c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:36.385683 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-17486961-43b5-427c-afbc-51edccddb3c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1340 microversion: 2.20 time: 0.154466 Aug 21 01:25:36.386660 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 45/88] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:36 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1340 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:36.466410 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-843e5173-a2bf-4dd1-8eb3-1ab904cb3f51 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:36.467458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-843e5173-a2bf-4dd1-8eb3-1ab904cb3f51 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:36.474036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-843e5173-a2bf-4dd1-8eb3-1ab904cb3f51 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:36.475372 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-843e5173-a2bf-4dd1-8eb3-1ab904cb3f51 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:36.476024 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-843e5173-a2bf-4dd1-8eb3-1ab904cb3f51 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:36.477042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-843e5173-a2bf-4dd1-8eb3-1ab904cb3f51 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:36.477909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-843e5173-a2bf-4dd1-8eb3-1ab904cb3f51 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:36.478507 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-843e5173-a2bf-4dd1-8eb3-1ab904cb3f51 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:36.545037 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-843e5173-a2bf-4dd1-8eb3-1ab904cb3f51 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:36.554153 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-843e5173-a2bf-4dd1-8eb3-1ab904cb3f51 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1296 microversion: 2.1 time: 0.091801 Aug 21 01:25:36.571398 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 44/89] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:36 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1296 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:36.579315 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-cff26da1-144d-4b96-8090-9885f1dd161c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:36.580264 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-cff26da1-144d-4b96-8090-9885f1dd161c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:36.593038 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cff26da1-144d-4b96-8090-9885f1dd161c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:36.593634 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cff26da1-144d-4b96-8090-9885f1dd161c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:36.594196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cff26da1-144d-4b96-8090-9885f1dd161c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:36.595560 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cff26da1-144d-4b96-8090-9885f1dd161c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:36.596353 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cff26da1-144d-4b96-8090-9885f1dd161c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:36.597007 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cff26da1-144d-4b96-8090-9885f1dd161c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:36.813557 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-cff26da1-144d-4b96-8090-9885f1dd161c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:36.824557 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-cff26da1-144d-4b96-8090-9885f1dd161c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.249137 Aug 21 01:25:36.834179 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 46/90] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:36 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:37.131485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d6a1c8e4-1c2a-42df-b7a9-e4fbd1ab7586 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:37.132092 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d6a1c8e4-1c2a-42df-b7a9-e4fbd1ab7586 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:37.138191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6a1c8e4-1c2a-42df-b7a9-e4fbd1ab7586 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:37.138673 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6a1c8e4-1c2a-42df-b7a9-e4fbd1ab7586 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:37.139109 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6a1c8e4-1c2a-42df-b7a9-e4fbd1ab7586 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:37.139986 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6a1c8e4-1c2a-42df-b7a9-e4fbd1ab7586 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:37.140479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6a1c8e4-1c2a-42df-b7a9-e4fbd1ab7586 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:37.140901 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6a1c8e4-1c2a-42df-b7a9-e4fbd1ab7586 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:37.337226 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d6a1c8e4-1c2a-42df-b7a9-e4fbd1ab7586 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:37.352300 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d6a1c8e4-1c2a-42df-b7a9-e4fbd1ab7586 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.223004 Aug 21 01:25:37.353700 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 45/91] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:37 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:37.415652 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8e1a41a2-d58c-4a5f-90c3-f0ffefebcac5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:37.418160 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8e1a41a2-d58c-4a5f-90c3-f0ffefebcac5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:37.430620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e1a41a2-d58c-4a5f-90c3-f0ffefebcac5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:37.431509 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e1a41a2-d58c-4a5f-90c3-f0ffefebcac5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:37.432302 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e1a41a2-d58c-4a5f-90c3-f0ffefebcac5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:37.433485 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e1a41a2-d58c-4a5f-90c3-f0ffefebcac5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:37.434227 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e1a41a2-d58c-4a5f-90c3-f0ffefebcac5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:37.435369 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e1a41a2-d58c-4a5f-90c3-f0ffefebcac5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:37.547828 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8e1a41a2-d58c-4a5f-90c3-f0ffefebcac5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:37.559066 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8e1a41a2-d58c-4a5f-90c3-f0ffefebcac5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:37.561021 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8e1a41a2-d58c-4a5f-90c3-f0ffefebcac5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:37.562366 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8e1a41a2-d58c-4a5f-90c3-f0ffefebcac5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1340 microversion: 2.20 time: 0.157201 Aug 21 01:25:37.563401 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 47/92] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:37 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1340 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:37.582763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-5692c0cb-8faf-476f-b977-79bfdcc1e161 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:37.584288 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-5692c0cb-8faf-476f-b977-79bfdcc1e161 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:37.592253 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5692c0cb-8faf-476f-b977-79bfdcc1e161 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:37.592967 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5692c0cb-8faf-476f-b977-79bfdcc1e161 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:37.593454 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5692c0cb-8faf-476f-b977-79bfdcc1e161 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:37.594170 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5692c0cb-8faf-476f-b977-79bfdcc1e161 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:37.594741 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5692c0cb-8faf-476f-b977-79bfdcc1e161 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:37.595219 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5692c0cb-8faf-476f-b977-79bfdcc1e161 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:37.600772 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=100 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:25:37.672047 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5692c0cb-8faf-476f-b977-79bfdcc1e161 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:37.710683 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5692c0cb-8faf-476f-b977-79bfdcc1e161 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1296 microversion: 2.1 time: 0.105347 Aug 21 01:25:37.710683 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 46/93] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:37 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1296 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:37.846774 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-eedc3415-0f95-45b0-b6bf-c39e80a44f6b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:37.847440 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-eedc3415-0f95-45b0-b6bf-c39e80a44f6b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:37.853962 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-eedc3415-0f95-45b0-b6bf-c39e80a44f6b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:37.854448 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-eedc3415-0f95-45b0-b6bf-c39e80a44f6b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:37.854889 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-eedc3415-0f95-45b0-b6bf-c39e80a44f6b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:37.855587 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-eedc3415-0f95-45b0-b6bf-c39e80a44f6b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:37.856041 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-eedc3415-0f95-45b0-b6bf-c39e80a44f6b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:37.856534 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-eedc3415-0f95-45b0-b6bf-c39e80a44f6b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:38.021917 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-eedc3415-0f95-45b0-b6bf-c39e80a44f6b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:38.031417 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-eedc3415-0f95-45b0-b6bf-c39e80a44f6b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.188644 Aug 21 01:25:38.032628 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 48/94] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:37 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:38.377526 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-22f95ea7-65bf-4438-960d-c4981d27fd90 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:38.379098 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-22f95ea7-65bf-4438-960d-c4981d27fd90 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:38.386554 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=38 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:25:38.388398 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-22f95ea7-65bf-4438-960d-c4981d27fd90 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:38.388958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-22f95ea7-65bf-4438-960d-c4981d27fd90 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:38.389652 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-22f95ea7-65bf-4438-960d-c4981d27fd90 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:38.390672 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-22f95ea7-65bf-4438-960d-c4981d27fd90 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:38.391437 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-22f95ea7-65bf-4438-960d-c4981d27fd90 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:38.392149 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-22f95ea7-65bf-4438-960d-c4981d27fd90 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:38.601659 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-166b877e-fd9a-4bff-9f4b-f808b4e10be0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:38.603097 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-166b877e-fd9a-4bff-9f4b-f808b4e10be0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:38.616141 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-166b877e-fd9a-4bff-9f4b-f808b4e10be0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:38.627909 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-166b877e-fd9a-4bff-9f4b-f808b4e10be0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:38.627909 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-166b877e-fd9a-4bff-9f4b-f808b4e10be0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:38.627909 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-166b877e-fd9a-4bff-9f4b-f808b4e10be0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:38.627909 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-166b877e-fd9a-4bff-9f4b-f808b4e10be0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:38.628464 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-166b877e-fd9a-4bff-9f4b-f808b4e10be0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:38.642941 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-22f95ea7-65bf-4438-960d-c4981d27fd90 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:38.664966 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-22f95ea7-65bf-4438-960d-c4981d27fd90 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.292286 Aug 21 01:25:38.665838 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 47/95] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:38 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:38.699949 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-166b877e-fd9a-4bff-9f4b-f808b4e10be0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:38.712152 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-166b877e-fd9a-4bff-9f4b-f808b4e10be0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:38.714538 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-166b877e-fd9a-4bff-9f4b-f808b4e10be0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:38.715662 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-166b877e-fd9a-4bff-9f4b-f808b4e10be0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1340 microversion: 2.20 time: 0.127650 Aug 21 01:25:38.717785 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 49/96] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:38 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1340 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:25:38.725260 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f9eb7741-a3af-426a-8710-19fca5d50a20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:38.726077 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f9eb7741-a3af-426a-8710-19fca5d50a20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:38.733104 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f9eb7741-a3af-426a-8710-19fca5d50a20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:38.733593 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f9eb7741-a3af-426a-8710-19fca5d50a20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:38.734227 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f9eb7741-a3af-426a-8710-19fca5d50a20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:38.734970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f9eb7741-a3af-426a-8710-19fca5d50a20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:38.735434 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f9eb7741-a3af-426a-8710-19fca5d50a20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:38.735890 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f9eb7741-a3af-426a-8710-19fca5d50a20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:38.821480 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f9eb7741-a3af-426a-8710-19fca5d50a20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:38.831443 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f9eb7741-a3af-426a-8710-19fca5d50a20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1296 microversion: 2.1 time: 0.111172 Aug 21 01:25:38.832053 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 48/97] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:38 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1296 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:39.053383 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a731647c-f576-4273-ab59-9685628d93a0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:39.054292 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a731647c-f576-4273-ab59-9685628d93a0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:39.071945 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a731647c-f576-4273-ab59-9685628d93a0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:39.072352 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a731647c-f576-4273-ab59-9685628d93a0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:39.072786 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a731647c-f576-4273-ab59-9685628d93a0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:39.073548 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a731647c-f576-4273-ab59-9685628d93a0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:39.074047 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a731647c-f576-4273-ab59-9685628d93a0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:39.074792 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a731647c-f576-4273-ab59-9685628d93a0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:39.344471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a731647c-f576-4273-ab59-9685628d93a0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:39.358957 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a731647c-f576-4273-ab59-9685628d93a0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.309633 Aug 21 01:25:39.362371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 50/98] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:39 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:39.689408 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f8e43927-563f-4762-a916-68aa9127b7e7 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:39.690043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f8e43927-563f-4762-a916-68aa9127b7e7 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:39.704951 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f8e43927-563f-4762-a916-68aa9127b7e7 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:39.704951 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f8e43927-563f-4762-a916-68aa9127b7e7 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:39.704951 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f8e43927-563f-4762-a916-68aa9127b7e7 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:39.704951 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f8e43927-563f-4762-a916-68aa9127b7e7 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:39.705554 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f8e43927-563f-4762-a916-68aa9127b7e7 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:39.705554 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f8e43927-563f-4762-a916-68aa9127b7e7 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:39.745527 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e19f1c50-75a3-4232-bf6c-0194d83d4358 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:39.747313 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e19f1c50-75a3-4232-bf6c-0194d83d4358 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:39.754821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e19f1c50-75a3-4232-bf6c-0194d83d4358 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:39.756685 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e19f1c50-75a3-4232-bf6c-0194d83d4358 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:39.758451 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e19f1c50-75a3-4232-bf6c-0194d83d4358 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:39.759434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e19f1c50-75a3-4232-bf6c-0194d83d4358 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:39.760127 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e19f1c50-75a3-4232-bf6c-0194d83d4358 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:39.760732 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e19f1c50-75a3-4232-bf6c-0194d83d4358 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:39.871944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e19f1c50-75a3-4232-bf6c-0194d83d4358 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:39.878830 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e19f1c50-75a3-4232-bf6c-0194d83d4358 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:39.881189 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e19f1c50-75a3-4232-bf6c-0194d83d4358 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:39.882306 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e19f1c50-75a3-4232-bf6c-0194d83d4358 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1340 microversion: 2.20 time: 0.141219 Aug 21 01:25:39.899847 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 51/99] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:39 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1340 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:39.918879 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f9015ec1-7c8d-4201-8835-887db7345cbe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:39.921092 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f9015ec1-7c8d-4201-8835-887db7345cbe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:39.928842 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f9015ec1-7c8d-4201-8835-887db7345cbe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:39.929591 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f9015ec1-7c8d-4201-8835-887db7345cbe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:39.930351 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f9015ec1-7c8d-4201-8835-887db7345cbe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:39.931219 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f9015ec1-7c8d-4201-8835-887db7345cbe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:39.931839 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f9015ec1-7c8d-4201-8835-887db7345cbe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:39.932444 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f9015ec1-7c8d-4201-8835-887db7345cbe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:39.947228 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f8e43927-563f-4762-a916-68aa9127b7e7 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:39.966926 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f8e43927-563f-4762-a916-68aa9127b7e7 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.281617 Aug 21 01:25:39.981151 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 49/100] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:39 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:40.020733 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f9015ec1-7c8d-4201-8835-887db7345cbe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:40.034044 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f9015ec1-7c8d-4201-8835-887db7345cbe tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1296 microversion: 2.1 time: 0.132227 Aug 21 01:25:40.035288 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 52/101] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:39 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1296 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:40.386759 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2a935b35-a908-4160-9592-f9bddc325fd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:40.386759 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2a935b35-a908-4160-9592-f9bddc325fd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:40.391496 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2a935b35-a908-4160-9592-f9bddc325fd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:40.391824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2a935b35-a908-4160-9592-f9bddc325fd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:40.392296 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2a935b35-a908-4160-9592-f9bddc325fd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:40.393006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2a935b35-a908-4160-9592-f9bddc325fd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:40.393464 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2a935b35-a908-4160-9592-f9bddc325fd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:40.393880 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2a935b35-a908-4160-9592-f9bddc325fd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:40.580819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2a935b35-a908-4160-9592-f9bddc325fd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:40.590806 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2a935b35-a908-4160-9592-f9bddc325fd7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.211731 Aug 21 01:25:40.591997 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 50/102] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:40 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:40.922557 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-23f9efd4-223a-4c9a-ba24-3159953e492f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:40.925509 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-23f9efd4-223a-4c9a-ba24-3159953e492f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:40.929304 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-23f9efd4-223a-4c9a-ba24-3159953e492f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:40.929788 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-23f9efd4-223a-4c9a-ba24-3159953e492f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:40.930228 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-23f9efd4-223a-4c9a-ba24-3159953e492f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:40.931055 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-23f9efd4-223a-4c9a-ba24-3159953e492f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:40.931504 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-23f9efd4-223a-4c9a-ba24-3159953e492f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:40.931928 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-23f9efd4-223a-4c9a-ba24-3159953e492f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:40.936431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=100 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:25:40.995678 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7a881198-e61f-4553-957a-aecea7095734 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:40.998320 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7a881198-e61f-4553-957a-aecea7095734 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:41.007160 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7a881198-e61f-4553-957a-aecea7095734 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:41.007844 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7a881198-e61f-4553-957a-aecea7095734 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:41.008457 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7a881198-e61f-4553-957a-aecea7095734 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:41.009318 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7a881198-e61f-4553-957a-aecea7095734 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:41.009960 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7a881198-e61f-4553-957a-aecea7095734 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:41.010531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7a881198-e61f-4553-957a-aecea7095734 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:41.059938 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-23f9efd4-223a-4c9a-ba24-3159953e492f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:41.069760 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-23f9efd4-223a-4c9a-ba24-3159953e492f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:41.072294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-23f9efd4-223a-4c9a-ba24-3159953e492f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:41.073643 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-23f9efd4-223a-4c9a-ba24-3159953e492f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1340 microversion: 2.20 time: 0.155304 Aug 21 01:25:41.074671 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 53/103] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:40 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1340 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:41.081118 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6be98669-175f-467d-a3c6-66b3f6de7f23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:41.081953 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6be98669-175f-467d-a3c6-66b3f6de7f23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:41.088725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6be98669-175f-467d-a3c6-66b3f6de7f23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:41.089223 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6be98669-175f-467d-a3c6-66b3f6de7f23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:41.089668 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6be98669-175f-467d-a3c6-66b3f6de7f23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:41.090373 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6be98669-175f-467d-a3c6-66b3f6de7f23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:41.090831 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6be98669-175f-467d-a3c6-66b3f6de7f23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:41.091282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6be98669-175f-467d-a3c6-66b3f6de7f23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:41.196377 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6be98669-175f-467d-a3c6-66b3f6de7f23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:41.206618 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6be98669-175f-467d-a3c6-66b3f6de7f23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1296 microversion: 2.1 time: 0.130588 Aug 21 01:25:41.207446 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 54/104] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:41 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1296 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:41.269447 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7a881198-e61f-4553-957a-aecea7095734 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:41.277800 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7a881198-e61f-4553-957a-aecea7095734 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.290579 Aug 21 01:25:41.279671 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 51/105] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:40 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:41.620591 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-623d3ca2-713d-4ebf-980a-42b78d1146d6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:41.621206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-623d3ca2-713d-4ebf-980a-42b78d1146d6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:41.638750 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-623d3ca2-713d-4ebf-980a-42b78d1146d6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:41.647163 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-623d3ca2-713d-4ebf-980a-42b78d1146d6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:41.647163 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-623d3ca2-713d-4ebf-980a-42b78d1146d6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:41.647163 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-623d3ca2-713d-4ebf-980a-42b78d1146d6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:41.647163 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-623d3ca2-713d-4ebf-980a-42b78d1146d6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:41.647611 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-623d3ca2-713d-4ebf-980a-42b78d1146d6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:41.808022 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-623d3ca2-713d-4ebf-980a-42b78d1146d6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:41.813860 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-623d3ca2-713d-4ebf-980a-42b78d1146d6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.203071 Aug 21 01:25:41.814429 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 55/106] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:41 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:42.100726 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2c380747-adb9-4edc-9836-f45230a12b11 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:42.101326 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2c380747-adb9-4edc-9836-f45230a12b11 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:42.107649 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c380747-adb9-4edc-9836-f45230a12b11 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:42.108163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c380747-adb9-4edc-9836-f45230a12b11 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:42.108605 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c380747-adb9-4edc-9836-f45230a12b11 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:42.109301 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c380747-adb9-4edc-9836-f45230a12b11 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:42.109761 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c380747-adb9-4edc-9836-f45230a12b11 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:42.110298 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c380747-adb9-4edc-9836-f45230a12b11 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:42.180658 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2c380747-adb9-4edc-9836-f45230a12b11 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:42.189350 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2c380747-adb9-4edc-9836-f45230a12b11 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:42.191075 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2c380747-adb9-4edc-9836-f45230a12b11 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:42.192320 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2c380747-adb9-4edc-9836-f45230a12b11 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1340 microversion: 2.20 time: 0.095359 Aug 21 01:25:42.192942 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 52/107] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:42 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1340 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:42.228007 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-bc6d5b07-8dd9-43ee-9418-831694bfbced tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:42.228677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-bc6d5b07-8dd9-43ee-9418-831694bfbced tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:42.235754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bc6d5b07-8dd9-43ee-9418-831694bfbced tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:42.235754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bc6d5b07-8dd9-43ee-9418-831694bfbced tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:42.236092 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bc6d5b07-8dd9-43ee-9418-831694bfbced tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:42.236843 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bc6d5b07-8dd9-43ee-9418-831694bfbced tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:42.237293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bc6d5b07-8dd9-43ee-9418-831694bfbced tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:42.237712 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bc6d5b07-8dd9-43ee-9418-831694bfbced tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:42.298974 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-91199871-75b5-4382-bdee-a90ee54f1f7d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:42.299764 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-91199871-75b5-4382-bdee-a90ee54f1f7d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:42.304376 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-bc6d5b07-8dd9-43ee-9418-831694bfbced tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:42.311273 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91199871-75b5-4382-bdee-a90ee54f1f7d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:42.311774 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91199871-75b5-4382-bdee-a90ee54f1f7d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:42.312260 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91199871-75b5-4382-bdee-a90ee54f1f7d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:42.312967 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91199871-75b5-4382-bdee-a90ee54f1f7d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:42.313424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91199871-75b5-4382-bdee-a90ee54f1f7d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:42.314052 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91199871-75b5-4382-bdee-a90ee54f1f7d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:42.315674 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-bc6d5b07-8dd9-43ee-9418-831694bfbced tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1296 microversion: 2.1 time: 0.091998 Aug 21 01:25:42.316381 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 56/108] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:42 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1296 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:42.450269 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-91199871-75b5-4382-bdee-a90ee54f1f7d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:42.458483 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-91199871-75b5-4382-bdee-a90ee54f1f7d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.163290 Aug 21 01:25:42.459040 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 53/109] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:42 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:42.834880 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-fd7977c7-1d4c-4acd-925f-903bde26f347 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:42.835572 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-fd7977c7-1d4c-4acd-925f-903bde26f347 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:42.837934 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:25:42.842979 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fd7977c7-1d4c-4acd-925f-903bde26f347 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:42.846507 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fd7977c7-1d4c-4acd-925f-903bde26f347 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:42.846507 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fd7977c7-1d4c-4acd-925f-903bde26f347 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:42.846507 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fd7977c7-1d4c-4acd-925f-903bde26f347 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:42.846507 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fd7977c7-1d4c-4acd-925f-903bde26f347 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:42.846842 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fd7977c7-1d4c-4acd-925f-903bde26f347 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:42.998122 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-fd7977c7-1d4c-4acd-925f-903bde26f347 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:43.007910 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-fd7977c7-1d4c-4acd-925f-903bde26f347 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.177298 Aug 21 01:25:43.008816 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 57/110] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:42 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:43.216414 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9c51a193-da63-45ba-b924-23c0f149fc16 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:43.218902 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9c51a193-da63-45ba-b924-23c0f149fc16 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:43.226134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c51a193-da63-45ba-b924-23c0f149fc16 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:43.227920 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c51a193-da63-45ba-b924-23c0f149fc16 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:43.228576 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c51a193-da63-45ba-b924-23c0f149fc16 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:43.229282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c51a193-da63-45ba-b924-23c0f149fc16 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:43.229978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c51a193-da63-45ba-b924-23c0f149fc16 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:43.230478 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c51a193-da63-45ba-b924-23c0f149fc16 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:43.250145 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=10 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:25:43.337998 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f7c2f924-ca6e-4cc1-8d92-9c75762ad224 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:43.340032 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f7c2f924-ca6e-4cc1-8d92-9c75762ad224 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:43.349693 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7c2f924-ca6e-4cc1-8d92-9c75762ad224 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:43.350690 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7c2f924-ca6e-4cc1-8d92-9c75762ad224 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:43.351315 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7c2f924-ca6e-4cc1-8d92-9c75762ad224 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:43.353159 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7c2f924-ca6e-4cc1-8d92-9c75762ad224 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:43.354454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7c2f924-ca6e-4cc1-8d92-9c75762ad224 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:43.355071 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7c2f924-ca6e-4cc1-8d92-9c75762ad224 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:43.420796 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9c51a193-da63-45ba-b924-23c0f149fc16 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:43.440437 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9c51a193-da63-45ba-b924-23c0f149fc16 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:43.444761 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9c51a193-da63-45ba-b924-23c0f149fc16 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:43.448688 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9c51a193-da63-45ba-b924-23c0f149fc16 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1340 microversion: 2.20 time: 0.236291 Aug 21 01:25:43.448688 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 54/111] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:43 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1340 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:25:43.479486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9a3611c6-5f56-4b60-9116-5989ad82e063 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:43.480127 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9a3611c6-5f56-4b60-9116-5989ad82e063 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:43.496376 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9a3611c6-5f56-4b60-9116-5989ad82e063 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:43.498373 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9a3611c6-5f56-4b60-9116-5989ad82e063 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:43.498819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9a3611c6-5f56-4b60-9116-5989ad82e063 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:43.499532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9a3611c6-5f56-4b60-9116-5989ad82e063 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:43.499993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9a3611c6-5f56-4b60-9116-5989ad82e063 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:43.500446 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9a3611c6-5f56-4b60-9116-5989ad82e063 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:43.648679 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f7c2f924-ca6e-4cc1-8d92-9c75762ad224 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:43.664472 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f7c2f924-ca6e-4cc1-8d92-9c75762ad224 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1296 microversion: 2.1 time: 0.332165 Aug 21 01:25:43.665827 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 58/112] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:43 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1296 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:44.037273 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-05fa64a0-ef31-4e2d-a94d-34db498fb91d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:44.038221 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-05fa64a0-ef31-4e2d-a94d-34db498fb91d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:44.049741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-05fa64a0-ef31-4e2d-a94d-34db498fb91d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:44.053463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-05fa64a0-ef31-4e2d-a94d-34db498fb91d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:44.054367 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-05fa64a0-ef31-4e2d-a94d-34db498fb91d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:44.055347 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-05fa64a0-ef31-4e2d-a94d-34db498fb91d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:44.055995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-05fa64a0-ef31-4e2d-a94d-34db498fb91d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:44.056692 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-05fa64a0-ef31-4e2d-a94d-34db498fb91d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:44.238589 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9a3611c6-5f56-4b60-9116-5989ad82e063 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:44.253784 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9a3611c6-5f56-4b60-9116-5989ad82e063 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.778800 Aug 21 01:25:44.254401 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 55/113] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:43 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 780 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:44.401434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-05fa64a0-ef31-4e2d-a94d-34db498fb91d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:44.411636 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-05fa64a0-ef31-4e2d-a94d-34db498fb91d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.377935 Aug 21 01:25:44.416493 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 59/114] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:44 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:44.469919 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-092fe367-d706-4856-80b5-fca1b3884ffb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:44.470454 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-092fe367-d706-4856-80b5-fca1b3884ffb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:44.476839 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-092fe367-d706-4856-80b5-fca1b3884ffb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:44.488335 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-092fe367-d706-4856-80b5-fca1b3884ffb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:44.488335 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-092fe367-d706-4856-80b5-fca1b3884ffb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:44.488335 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-092fe367-d706-4856-80b5-fca1b3884ffb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:44.488335 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-092fe367-d706-4856-80b5-fca1b3884ffb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:44.489258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-092fe367-d706-4856-80b5-fca1b3884ffb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:44.694674 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6700e621-4852-4ee4-bb81-c7f5e38f4022 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:44.695281 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6700e621-4852-4ee4-bb81-c7f5e38f4022 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:44.702582 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6700e621-4852-4ee4-bb81-c7f5e38f4022 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:44.703428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6700e621-4852-4ee4-bb81-c7f5e38f4022 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:44.705551 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6700e621-4852-4ee4-bb81-c7f5e38f4022 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:44.714161 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6700e621-4852-4ee4-bb81-c7f5e38f4022 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:44.714161 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6700e621-4852-4ee4-bb81-c7f5e38f4022 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:44.714161 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6700e621-4852-4ee4-bb81-c7f5e38f4022 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:44.797562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-092fe367-d706-4856-80b5-fca1b3884ffb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:44.807240 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-092fe367-d706-4856-80b5-fca1b3884ffb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:44.811134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-092fe367-d706-4856-80b5-fca1b3884ffb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:44.811134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-092fe367-d706-4856-80b5-fca1b3884ffb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1338 microversion: 2.20 time: 0.344459 Aug 21 01:25:44.811134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 56/115] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:44 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1338 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:44.934727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6700e621-4852-4ee4-bb81-c7f5e38f4022 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:44.955955 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6700e621-4852-4ee4-bb81-c7f5e38f4022 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.265711 Aug 21 01:25:44.956592 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 60/116] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:44 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:45.305201 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fdf2ed6d-74ba-4596-a185-abb6d7ea64f1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:45.306657 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fdf2ed6d-74ba-4596-a185-abb6d7ea64f1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:45.319461 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fdf2ed6d-74ba-4596-a185-abb6d7ea64f1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:45.320492 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fdf2ed6d-74ba-4596-a185-abb6d7ea64f1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:45.321394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fdf2ed6d-74ba-4596-a185-abb6d7ea64f1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:45.322834 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fdf2ed6d-74ba-4596-a185-abb6d7ea64f1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:45.323752 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fdf2ed6d-74ba-4596-a185-abb6d7ea64f1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:45.324634 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fdf2ed6d-74ba-4596-a185-abb6d7ea64f1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:45.438796 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5677065a-2739-4443-bd38-52602d5550ca tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:45.440763 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5677065a-2739-4443-bd38-52602d5550ca tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:45.460896 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5677065a-2739-4443-bd38-52602d5550ca tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:45.461762 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5677065a-2739-4443-bd38-52602d5550ca tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:45.470739 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5677065a-2739-4443-bd38-52602d5550ca tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:45.473858 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5677065a-2739-4443-bd38-52602d5550ca tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:45.475567 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5677065a-2739-4443-bd38-52602d5550ca tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:45.478502 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5677065a-2739-4443-bd38-52602d5550ca tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:46.149239 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5677065a-2739-4443-bd38-52602d5550ca tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:46.173511 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5677065a-2739-4443-bd38-52602d5550ca tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.744514 Aug 21 01:25:46.174103 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 61/117] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:45 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 746 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:46.180049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e7bd934b-dcac-40b0-ab9e-65ceb056c7eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:46.196925 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e7bd934b-dcac-40b0-ab9e-65ceb056c7eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:46.212960 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e7bd934b-dcac-40b0-ab9e-65ceb056c7eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:46.213452 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e7bd934b-dcac-40b0-ab9e-65ceb056c7eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:46.214760 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e7bd934b-dcac-40b0-ab9e-65ceb056c7eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:46.216865 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fdf2ed6d-74ba-4596-a185-abb6d7ea64f1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:46.222891 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e7bd934b-dcac-40b0-ab9e-65ceb056c7eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:46.230707 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e7bd934b-dcac-40b0-ab9e-65ceb056c7eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:46.231482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fdf2ed6d-74ba-4596-a185-abb6d7ea64f1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.934327 Aug 21 01:25:46.231627 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e7bd934b-dcac-40b0-ab9e-65ceb056c7eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:46.232982 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 57/118] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:45 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 938 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:46.254703 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-05a25603-6c6d-437a-9e1d-7e2d3cbd3128 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:46.255375 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-05a25603-6c6d-437a-9e1d-7e2d3cbd3128 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:46.279348 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-05a25603-6c6d-437a-9e1d-7e2d3cbd3128 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:46.279858 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-05a25603-6c6d-437a-9e1d-7e2d3cbd3128 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:46.281177 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-05a25603-6c6d-437a-9e1d-7e2d3cbd3128 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:46.284802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-05a25603-6c6d-437a-9e1d-7e2d3cbd3128 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:46.285311 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-05a25603-6c6d-437a-9e1d-7e2d3cbd3128 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:46.286276 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-05a25603-6c6d-437a-9e1d-7e2d3cbd3128 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:46.382338 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e7bd934b-dcac-40b0-ab9e-65ceb056c7eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:46.393974 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e7bd934b-dcac-40b0-ab9e-65ceb056c7eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:46.399558 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e7bd934b-dcac-40b0-ab9e-65ceb056c7eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:46.403853 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e7bd934b-dcac-40b0-ab9e-65ceb056c7eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1338 microversion: 2.20 time: 0.226240 Aug 21 01:25:46.409926 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 62/119] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:46 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1338 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:46.900688 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-05a25603-6c6d-437a-9e1d-7e2d3cbd3128 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:46.920549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-05a25603-6c6d-437a-9e1d-7e2d3cbd3128 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.686103 Aug 21 01:25:46.921365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 58/120] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:46 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:47.086858 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-1a69d46f-2afb-44e2-bc3a-6dbfd1f151bb req-171386f0-f900-463b-8a6e-b403d0bef588 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c266a09a-3f81-4111-a987-b1f4be65cdc4", "tag": "8d6fe117-8bd6-47dc-9dd9-1b92667e8ba7"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:47.105860 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-1a69d46f-2afb-44e2-bc3a-6dbfd1f151bb req-171386f0-f900-463b-8a6e-b403d0bef588 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:47.108238 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-1a69d46f-2afb-44e2-bc3a-6dbfd1f151bb req-171386f0-f900-463b-8a6e-b403d0bef588 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:47.109145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-1a69d46f-2afb-44e2-bc3a-6dbfd1f151bb req-171386f0-f900-463b-8a6e-b403d0bef588 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:47.131322 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-1a69d46f-2afb-44e2-bc3a-6dbfd1f151bb req-171386f0-f900-463b-8a6e-b403d0bef588 service nova] Creating event network-changed:8d6fe117-8bd6-47dc-9dd9-1b92667e8ba7 for instance c266a09a-3f81-4111-a987-b1f4be65cdc4 on hv-cinder-maste Aug 21 01:25:47.148253 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-1a69d46f-2afb-44e2-bc3a-6dbfd1f151bb req-171386f0-f900-463b-8a6e-b403d0bef588 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065933 Aug 21 01:25:47.149180 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 63/121] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:25:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:25:47.203609 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2c5525ae-c2be-4cb1-a73b-5b3094afbc94 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:47.203609 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2c5525ae-c2be-4cb1-a73b-5b3094afbc94 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:47.213622 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c5525ae-c2be-4cb1-a73b-5b3094afbc94 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:47.214269 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c5525ae-c2be-4cb1-a73b-5b3094afbc94 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:47.215462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c5525ae-c2be-4cb1-a73b-5b3094afbc94 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:47.216589 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c5525ae-c2be-4cb1-a73b-5b3094afbc94 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:47.217563 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c5525ae-c2be-4cb1-a73b-5b3094afbc94 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:47.218164 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c5525ae-c2be-4cb1-a73b-5b3094afbc94 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:47.256975 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b6cc4d94-4738-4a19-8247-cad5998cab1a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:47.257908 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b6cc4d94-4738-4a19-8247-cad5998cab1a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:47.265269 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b6cc4d94-4738-4a19-8247-cad5998cab1a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:47.272464 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b6cc4d94-4738-4a19-8247-cad5998cab1a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:47.272464 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b6cc4d94-4738-4a19-8247-cad5998cab1a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:47.272464 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b6cc4d94-4738-4a19-8247-cad5998cab1a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:47.272464 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b6cc4d94-4738-4a19-8247-cad5998cab1a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:47.273006 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b6cc4d94-4738-4a19-8247-cad5998cab1a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:47.572975 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2c5525ae-c2be-4cb1-a73b-5b3094afbc94 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:47.581598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2c5525ae-c2be-4cb1-a73b-5b3094afbc94 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.390361 Aug 21 01:25:47.582192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 59/122] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:47 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:47.600017 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c66ad312-47b9-4271-b11f-c42e0b1e9c59 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:47.608265 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c66ad312-47b9-4271-b11f-c42e0b1e9c59 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:47.608265 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c66ad312-47b9-4271-b11f-c42e0b1e9c59 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:47.608767 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c66ad312-47b9-4271-b11f-c42e0b1e9c59 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:47.608906 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c66ad312-47b9-4271-b11f-c42e0b1e9c59 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:47.612432 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b6cc4d94-4738-4a19-8247-cad5998cab1a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:47.618255 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c66ad312-47b9-4271-b11f-c42e0b1e9c59 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:47.619767 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c66ad312-47b9-4271-b11f-c42e0b1e9c59 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:47.621419 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c66ad312-47b9-4271-b11f-c42e0b1e9c59 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:47.642822 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b6cc4d94-4738-4a19-8247-cad5998cab1a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.387811 Aug 21 01:25:47.642822 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 64/123] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:47 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:47.842224 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c66ad312-47b9-4271-b11f-c42e0b1e9c59 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:47.853950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c66ad312-47b9-4271-b11f-c42e0b1e9c59 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:47.855759 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c66ad312-47b9-4271-b11f-c42e0b1e9c59 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:47.866034 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c66ad312-47b9-4271-b11f-c42e0b1e9c59 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.270476 Aug 21 01:25:47.867031 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 60/124] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:47 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:47.949102 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-52eb83cf-7b7e-4053-8ee4-f3d962a70c71 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:47.950124 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-52eb83cf-7b7e-4053-8ee4-f3d962a70c71 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:47.958199 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-52eb83cf-7b7e-4053-8ee4-f3d962a70c71 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:47.959655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-52eb83cf-7b7e-4053-8ee4-f3d962a70c71 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:47.960780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-52eb83cf-7b7e-4053-8ee4-f3d962a70c71 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:47.962061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-52eb83cf-7b7e-4053-8ee4-f3d962a70c71 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:47.963042 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-52eb83cf-7b7e-4053-8ee4-f3d962a70c71 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:47.963799 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-52eb83cf-7b7e-4053-8ee4-f3d962a70c71 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:48.149219 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-52eb83cf-7b7e-4053-8ee4-f3d962a70c71 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:48.157152 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-52eb83cf-7b7e-4053-8ee4-f3d962a70c71 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.212347 Aug 21 01:25:48.157741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 65/125] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:47 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:48.611987 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-95ce226d-d403-41af-b7a0-5e79f52f80ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:48.611987 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-95ce226d-d403-41af-b7a0-5e79f52f80ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:48.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-95ce226d-d403-41af-b7a0-5e79f52f80ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:48.614688 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-95ce226d-d403-41af-b7a0-5e79f52f80ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:48.615166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-95ce226d-d403-41af-b7a0-5e79f52f80ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:48.615880 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-95ce226d-d403-41af-b7a0-5e79f52f80ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:48.616371 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-95ce226d-d403-41af-b7a0-5e79f52f80ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:48.616806 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-95ce226d-d403-41af-b7a0-5e79f52f80ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:48.669750 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e26b9984-90f1-457a-b589-e2c21952d98a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:48.670471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e26b9984-90f1-457a-b589-e2c21952d98a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:48.676911 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e26b9984-90f1-457a-b589-e2c21952d98a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:48.677664 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e26b9984-90f1-457a-b589-e2c21952d98a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:48.678547 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e26b9984-90f1-457a-b589-e2c21952d98a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:48.679544 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e26b9984-90f1-457a-b589-e2c21952d98a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:48.680189 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e26b9984-90f1-457a-b589-e2c21952d98a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:48.680785 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e26b9984-90f1-457a-b589-e2c21952d98a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:49.126137 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-95ce226d-d403-41af-b7a0-5e79f52f80ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:49.142238 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-95ce226d-d403-41af-b7a0-5e79f52f80ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.542995 Aug 21 01:25:49.142851 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 61/126] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:48 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:49.148579 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-432f3d42-2425-4f73-aecf-0b70af75f732 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:49.149701 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-432f3d42-2425-4f73-aecf-0b70af75f732 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:49.156451 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-432f3d42-2425-4f73-aecf-0b70af75f732 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:49.165005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-432f3d42-2425-4f73-aecf-0b70af75f732 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:49.165005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-432f3d42-2425-4f73-aecf-0b70af75f732 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:49.165005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-432f3d42-2425-4f73-aecf-0b70af75f732 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:49.165005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-432f3d42-2425-4f73-aecf-0b70af75f732 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:49.165869 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-432f3d42-2425-4f73-aecf-0b70af75f732 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:49.584963 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e26b9984-90f1-457a-b589-e2c21952d98a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:49.596928 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e26b9984-90f1-457a-b589-e2c21952d98a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.930930 Aug 21 01:25:49.597524 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 66/127] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:48 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 932 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:49.611930 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-991101cf-45cd-47d6-a354-0cc7471c02c8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:49.613009 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-991101cf-45cd-47d6-a354-0cc7471c02c8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:49.632188 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-991101cf-45cd-47d6-a354-0cc7471c02c8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:49.632810 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-991101cf-45cd-47d6-a354-0cc7471c02c8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:49.633244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-991101cf-45cd-47d6-a354-0cc7471c02c8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:49.633950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-991101cf-45cd-47d6-a354-0cc7471c02c8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:49.634470 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-991101cf-45cd-47d6-a354-0cc7471c02c8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:49.634892 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-991101cf-45cd-47d6-a354-0cc7471c02c8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:50.088043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-432f3d42-2425-4f73-aecf-0b70af75f732 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:50.095561 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=100 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:25:50.100513 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-432f3d42-2425-4f73-aecf-0b70af75f732 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:50.105073 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-432f3d42-2425-4f73-aecf-0b70af75f732 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:50.107267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-432f3d42-2425-4f73-aecf-0b70af75f732 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.963095 Aug 21 01:25:50.126151 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 62/128] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:49 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 982 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:50.249309 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-e965a1e5-c53a-4291-b089-15278be4587f req-0394f843-b539-4167-9681-c020943c406c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "90e88110-1175-4ccf-bf6b-8efb43ab945e", "name": "network-vif-plugged", "status": "completed", "tag": "f56acc10-3115-43fe-929a-cb8d30842aef"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:50.264621 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-e965a1e5-c53a-4291-b089-15278be4587f req-0394f843-b539-4167-9681-c020943c406c service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:50.265936 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-e965a1e5-c53a-4291-b089-15278be4587f req-0394f843-b539-4167-9681-c020943c406c service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:50.266614 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-e965a1e5-c53a-4291-b089-15278be4587f req-0394f843-b539-4167-9681-c020943c406c service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:50.296182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-e965a1e5-c53a-4291-b089-15278be4587f req-0394f843-b539-4167-9681-c020943c406c service nova] Creating event network-vif-plugged:f56acc10-3115-43fe-929a-cb8d30842aef for instance 90e88110-1175-4ccf-bf6b-8efb43ab945e on hv-cinder-maste Aug 21 01:25:50.302817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-e965a1e5-c53a-4291-b089-15278be4587f req-0394f843-b539-4167-9681-c020943c406c service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.139956 Aug 21 01:25:50.303572 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 63/129] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:25:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:25:50.309918 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-91d16f29-eb4c-4c5d-9c62-5704ef19adb6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:50.310807 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-91d16f29-eb4c-4c5d-9c62-5704ef19adb6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:50.319008 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91d16f29-eb4c-4c5d-9c62-5704ef19adb6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:50.319830 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91d16f29-eb4c-4c5d-9c62-5704ef19adb6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:50.320809 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91d16f29-eb4c-4c5d-9c62-5704ef19adb6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:50.321585 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91d16f29-eb4c-4c5d-9c62-5704ef19adb6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:50.322130 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91d16f29-eb4c-4c5d-9c62-5704ef19adb6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:50.322787 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91d16f29-eb4c-4c5d-9c62-5704ef19adb6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:50.334875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-991101cf-45cd-47d6-a354-0cc7471c02c8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:50.373470 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-991101cf-45cd-47d6-a354-0cc7471c02c8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.774313 Aug 21 01:25:50.388441 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 67/130] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:49 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 790 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:50.621295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-238a742d-2074-40ae-b855-5d2b5dbc769a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:50.623310 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-238a742d-2074-40ae-b855-5d2b5dbc769a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:50.645206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-238a742d-2074-40ae-b855-5d2b5dbc769a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:50.645206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-238a742d-2074-40ae-b855-5d2b5dbc769a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:50.645206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-238a742d-2074-40ae-b855-5d2b5dbc769a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:50.645206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-238a742d-2074-40ae-b855-5d2b5dbc769a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:50.645594 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-238a742d-2074-40ae-b855-5d2b5dbc769a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:50.645594 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-238a742d-2074-40ae-b855-5d2b5dbc769a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:51.064791 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-91d16f29-eb4c-4c5d-9c62-5704ef19adb6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:51.092248 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-91d16f29-eb4c-4c5d-9c62-5704ef19adb6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.771130 Aug 21 01:25:51.092248 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 64/131] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:50 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 773 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:51.144029 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-56e8477f-4f7a-4e82-86f7-2cb48b434b2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:51.155627 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-56e8477f-4f7a-4e82-86f7-2cb48b434b2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:51.169711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-56e8477f-4f7a-4e82-86f7-2cb48b434b2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:51.170856 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-56e8477f-4f7a-4e82-86f7-2cb48b434b2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:51.196644 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-56e8477f-4f7a-4e82-86f7-2cb48b434b2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:51.196644 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-56e8477f-4f7a-4e82-86f7-2cb48b434b2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:51.196644 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-56e8477f-4f7a-4e82-86f7-2cb48b434b2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:51.196644 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-56e8477f-4f7a-4e82-86f7-2cb48b434b2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:51.257383 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-238a742d-2074-40ae-b855-5d2b5dbc769a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:51.268301 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-238a742d-2074-40ae-b855-5d2b5dbc769a tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.652459 Aug 21 01:25:51.286124 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 68/132] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:50 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 671 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:51.395533 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7b80b90c-209d-454d-acd9-ceca01c892e8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:51.396426 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7b80b90c-209d-454d-acd9-ceca01c892e8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:51.405354 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7b80b90c-209d-454d-acd9-ceca01c892e8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:51.405961 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7b80b90c-209d-454d-acd9-ceca01c892e8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:51.406498 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7b80b90c-209d-454d-acd9-ceca01c892e8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:51.407305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7b80b90c-209d-454d-acd9-ceca01c892e8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:51.407859 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7b80b90c-209d-454d-acd9-ceca01c892e8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:51.408521 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7b80b90c-209d-454d-acd9-ceca01c892e8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:51.472916 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-56e8477f-4f7a-4e82-86f7-2cb48b434b2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:51.498739 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-56e8477f-4f7a-4e82-86f7-2cb48b434b2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:51.505473 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-56e8477f-4f7a-4e82-86f7-2cb48b434b2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:51.505473 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-56e8477f-4f7a-4e82-86f7-2cb48b434b2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.371234 Aug 21 01:25:51.506591 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 65/133] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:51 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:51.763359 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7b80b90c-209d-454d-acd9-ceca01c892e8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:51.783288 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7b80b90c-209d-454d-acd9-ceca01c892e8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.387075 Aug 21 01:25:51.784717 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 69/134] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:51 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:52.082876 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-cd80b712-f9e1-46f0-b515-1913c47abb50 req-12387e54-ce39-45b5-bb49-c01c80294c94 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1", "tag": "9a09b4f3-32e5-4362-82ef-ab73b634b1bf"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:52.103366 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-174cda30-ce72-4fe0-81d5-6724842cc1c6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:52.106980 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-174cda30-ce72-4fe0-81d5-6724842cc1c6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:52.118332 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-174cda30-ce72-4fe0-81d5-6724842cc1c6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:52.119143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-174cda30-ce72-4fe0-81d5-6724842cc1c6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:52.119794 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-174cda30-ce72-4fe0-81d5-6724842cc1c6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:52.120935 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-174cda30-ce72-4fe0-81d5-6724842cc1c6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:52.123325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-174cda30-ce72-4fe0-81d5-6724842cc1c6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:52.124271 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-174cda30-ce72-4fe0-81d5-6724842cc1c6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:52.152831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-cd80b712-f9e1-46f0-b515-1913c47abb50 req-12387e54-ce39-45b5-bb49-c01c80294c94 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:52.154422 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-cd80b712-f9e1-46f0-b515-1913c47abb50 req-12387e54-ce39-45b5-bb49-c01c80294c94 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:52.157820 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-cd80b712-f9e1-46f0-b515-1913c47abb50 req-12387e54-ce39-45b5-bb49-c01c80294c94 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:52.165272 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=100 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:25:52.185163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-cd80b712-f9e1-46f0-b515-1913c47abb50 req-12387e54-ce39-45b5-bb49-c01c80294c94 service nova] Creating event network-changed:9a09b4f3-32e5-4362-82ef-ab73b634b1bf for instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 on hv-cinder-maste Aug 21 01:25:52.199618 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-cd80b712-f9e1-46f0-b515-1913c47abb50 req-12387e54-ce39-45b5-bb49-c01c80294c94 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.120120 Aug 21 01:25:52.200721 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 66/135] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:25:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:52.299677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4860fcce-d925-4911-959c-edfb9383ff00 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:52.301453 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4860fcce-d925-4911-959c-edfb9383ff00 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:52.310366 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4860fcce-d925-4911-959c-edfb9383ff00 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:52.311085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4860fcce-d925-4911-959c-edfb9383ff00 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:52.311911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4860fcce-d925-4911-959c-edfb9383ff00 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:52.312990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4860fcce-d925-4911-959c-edfb9383ff00 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:52.313665 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4860fcce-d925-4911-959c-edfb9383ff00 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:52.314298 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4860fcce-d925-4911-959c-edfb9383ff00 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:52.329281 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-174cda30-ce72-4fe0-81d5-6724842cc1c6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:52.334974 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-174cda30-ce72-4fe0-81d5-6724842cc1c6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.237853 Aug 21 01:25:52.335989 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 70/136] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:52 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:52.467630 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4860fcce-d925-4911-959c-edfb9383ff00 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:52.477287 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4860fcce-d925-4911-959c-edfb9383ff00 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.182803 Aug 21 01:25:52.477941 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 67/137] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:52 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:52.528632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d382fc1d-4305-48aa-b821-48ed3eb40687 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:52.530923 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d382fc1d-4305-48aa-b821-48ed3eb40687 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:52.536389 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d382fc1d-4305-48aa-b821-48ed3eb40687 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:52.536877 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d382fc1d-4305-48aa-b821-48ed3eb40687 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:52.537315 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d382fc1d-4305-48aa-b821-48ed3eb40687 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:52.538264 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d382fc1d-4305-48aa-b821-48ed3eb40687 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:52.538716 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d382fc1d-4305-48aa-b821-48ed3eb40687 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:52.539159 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d382fc1d-4305-48aa-b821-48ed3eb40687 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:52.765544 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d382fc1d-4305-48aa-b821-48ed3eb40687 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:52.776420 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d382fc1d-4305-48aa-b821-48ed3eb40687 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:52.778155 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d382fc1d-4305-48aa-b821-48ed3eb40687 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:52.779478 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d382fc1d-4305-48aa-b821-48ed3eb40687 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.255304 Aug 21 01:25:52.780133 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 71/138] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:52 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:52.802424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-78079518-20a6-4f1d-8da5-724124b7e6af tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:52.803061 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-78079518-20a6-4f1d-8da5-724124b7e6af tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:52.809436 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-78079518-20a6-4f1d-8da5-724124b7e6af tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:52.810195 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-78079518-20a6-4f1d-8da5-724124b7e6af tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:52.810799 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-78079518-20a6-4f1d-8da5-724124b7e6af tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:52.811529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-78079518-20a6-4f1d-8da5-724124b7e6af tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:52.811982 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-78079518-20a6-4f1d-8da5-724124b7e6af tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:52.812436 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-78079518-20a6-4f1d-8da5-724124b7e6af tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:52.961278 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-78079518-20a6-4f1d-8da5-724124b7e6af tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:52.969740 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-78079518-20a6-4f1d-8da5-724124b7e6af tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.171398 Aug 21 01:25:52.970426 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 68/139] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:52 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:53.364079 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-932c69fd-0685-4f8e-83a2-1fced703b92a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:53.364785 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-932c69fd-0685-4f8e-83a2-1fced703b92a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:53.371248 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-932c69fd-0685-4f8e-83a2-1fced703b92a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:53.371734 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-932c69fd-0685-4f8e-83a2-1fced703b92a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:53.372199 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-932c69fd-0685-4f8e-83a2-1fced703b92a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:53.372919 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-932c69fd-0685-4f8e-83a2-1fced703b92a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:53.373444 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-932c69fd-0685-4f8e-83a2-1fced703b92a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:53.373878 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-932c69fd-0685-4f8e-83a2-1fced703b92a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:53.498734 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-882e6c24-f780-437a-a9ab-060e7249b725 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:53.499417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-882e6c24-f780-437a-a9ab-060e7249b725 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:53.507106 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-882e6c24-f780-437a-a9ab-060e7249b725 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:53.507838 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-882e6c24-f780-437a-a9ab-060e7249b725 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:53.508386 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-882e6c24-f780-437a-a9ab-060e7249b725 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:53.509189 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-882e6c24-f780-437a-a9ab-060e7249b725 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:53.509800 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-882e6c24-f780-437a-a9ab-060e7249b725 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:53.510376 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-882e6c24-f780-437a-a9ab-060e7249b725 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:53.521293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-932c69fd-0685-4f8e-83a2-1fced703b92a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:53.535637 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-932c69fd-0685-4f8e-83a2-1fced703b92a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.178934 Aug 21 01:25:53.536310 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 72/140] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:53 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:53.668531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-882e6c24-f780-437a-a9ab-060e7249b725 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:53.676935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-882e6c24-f780-437a-a9ab-060e7249b725 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.182466 Aug 21 01:25:53.679044 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 69/141] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:53 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:53.802736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-25b4dcab-740d-4711-a020-f6135c1b91ed tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:53.803287 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-25b4dcab-740d-4711-a020-f6135c1b91ed tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:53.809783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-25b4dcab-740d-4711-a020-f6135c1b91ed tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:53.810208 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-25b4dcab-740d-4711-a020-f6135c1b91ed tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:53.810638 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-25b4dcab-740d-4711-a020-f6135c1b91ed tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:53.811330 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-25b4dcab-740d-4711-a020-f6135c1b91ed tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:53.811776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-25b4dcab-740d-4711-a020-f6135c1b91ed tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:53.812505 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-25b4dcab-740d-4711-a020-f6135c1b91ed tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:53.964492 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-25b4dcab-740d-4711-a020-f6135c1b91ed tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:53.974953 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-25b4dcab-740d-4711-a020-f6135c1b91ed tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:53.977573 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-25b4dcab-740d-4711-a020-f6135c1b91ed tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:53.979669 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-25b4dcab-740d-4711-a020-f6135c1b91ed tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.181273 Aug 21 01:25:53.980894 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 73/142] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:53 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:53.994637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b5b955ae-6fca-4e4a-8a28-79e0986f59d7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:53.994637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-b5b955ae-6fca-4e4a-8a28-79e0986f59d7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:53.999898 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b5b955ae-6fca-4e4a-8a28-79e0986f59d7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:54.000800 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b5b955ae-6fca-4e4a-8a28-79e0986f59d7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:54.001249 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b5b955ae-6fca-4e4a-8a28-79e0986f59d7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:54.001963 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b5b955ae-6fca-4e4a-8a28-79e0986f59d7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:54.002420 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b5b955ae-6fca-4e4a-8a28-79e0986f59d7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:54.002848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b5b955ae-6fca-4e4a-8a28-79e0986f59d7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:54.144515 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b5b955ae-6fca-4e4a-8a28-79e0986f59d7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:54.153129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b5b955ae-6fca-4e4a-8a28-79e0986f59d7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.165615 Aug 21 01:25:54.153784 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 70/143] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:53 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:54.557056 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8d679ba3-37bc-451a-8de4-be83ccf76769 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:54.557656 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8d679ba3-37bc-451a-8de4-be83ccf76769 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:54.563588 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d679ba3-37bc-451a-8de4-be83ccf76769 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:54.564064 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d679ba3-37bc-451a-8de4-be83ccf76769 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:54.564592 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d679ba3-37bc-451a-8de4-be83ccf76769 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:54.565267 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d679ba3-37bc-451a-8de4-be83ccf76769 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:54.565714 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d679ba3-37bc-451a-8de4-be83ccf76769 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:54.566185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d679ba3-37bc-451a-8de4-be83ccf76769 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:54.698763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-185125a8-c48a-4c90-a489-c185d663317c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:54.699366 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-185125a8-c48a-4c90-a489-c185d663317c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:54.705928 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-185125a8-c48a-4c90-a489-c185d663317c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:54.707255 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-185125a8-c48a-4c90-a489-c185d663317c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:54.707947 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-185125a8-c48a-4c90-a489-c185d663317c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:54.708698 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-185125a8-c48a-4c90-a489-c185d663317c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:54.709160 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-185125a8-c48a-4c90-a489-c185d663317c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:54.709586 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-185125a8-c48a-4c90-a489-c185d663317c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:54.713594 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8d679ba3-37bc-451a-8de4-be83ccf76769 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:54.721520 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8d679ba3-37bc-451a-8de4-be83ccf76769 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.168261 Aug 21 01:25:54.722102 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 74/144] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:54 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:54.854189 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-185125a8-c48a-4c90-a489-c185d663317c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:54.862581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-185125a8-c48a-4c90-a489-c185d663317c tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.167498 Aug 21 01:25:54.863200 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 71/145] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:54 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:55.005198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-70713642-36f2-42e3-afe9-b767a54b6453 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:55.005966 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-70713642-36f2-42e3-afe9-b767a54b6453 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:55.012549 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-70713642-36f2-42e3-afe9-b767a54b6453 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:55.013032 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-70713642-36f2-42e3-afe9-b767a54b6453 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:55.013472 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-70713642-36f2-42e3-afe9-b767a54b6453 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:55.014175 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-70713642-36f2-42e3-afe9-b767a54b6453 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:55.014628 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-70713642-36f2-42e3-afe9-b767a54b6453 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:55.015096 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-70713642-36f2-42e3-afe9-b767a54b6453 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:55.153133 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-70713642-36f2-42e3-afe9-b767a54b6453 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:55.161313 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-70713642-36f2-42e3-afe9-b767a54b6453 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:55.163855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-70713642-36f2-42e3-afe9-b767a54b6453 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:55.165198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-70713642-36f2-42e3-afe9-b767a54b6453 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.164455 Aug 21 01:25:55.165749 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 75/146] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:54 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:55.181397 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-57ae141e-cc31-4434-bb23-0042813b36c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:55.181900 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-57ae141e-cc31-4434-bb23-0042813b36c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:55.188062 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-57ae141e-cc31-4434-bb23-0042813b36c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:55.194312 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-57ae141e-cc31-4434-bb23-0042813b36c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:55.194312 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-57ae141e-cc31-4434-bb23-0042813b36c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:55.194312 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-57ae141e-cc31-4434-bb23-0042813b36c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:55.194312 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-57ae141e-cc31-4434-bb23-0042813b36c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:55.194638 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-57ae141e-cc31-4434-bb23-0042813b36c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:55.331607 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-57ae141e-cc31-4434-bb23-0042813b36c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:55.339422 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-57ae141e-cc31-4434-bb23-0042813b36c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.164735 Aug 21 01:25:55.343787 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 72/147] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:55 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:55.743077 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6dddc720-0be5-4d78-9963-f2fc5ee19009 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:55.743791 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6dddc720-0be5-4d78-9963-f2fc5ee19009 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:55.750006 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6dddc720-0be5-4d78-9963-f2fc5ee19009 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:55.750504 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6dddc720-0be5-4d78-9963-f2fc5ee19009 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:55.751025 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6dddc720-0be5-4d78-9963-f2fc5ee19009 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:55.751828 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6dddc720-0be5-4d78-9963-f2fc5ee19009 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:55.752199 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6dddc720-0be5-4d78-9963-f2fc5ee19009 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:55.752632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6dddc720-0be5-4d78-9963-f2fc5ee19009 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:55.883223 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8737e681-5d6a-4bc7-a280-c05bf4ae21ea tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:55.883935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8737e681-5d6a-4bc7-a280-c05bf4ae21ea tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:55.890740 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8737e681-5d6a-4bc7-a280-c05bf4ae21ea tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:55.891258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8737e681-5d6a-4bc7-a280-c05bf4ae21ea tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:55.894233 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8737e681-5d6a-4bc7-a280-c05bf4ae21ea tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:55.894233 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8737e681-5d6a-4bc7-a280-c05bf4ae21ea tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:55.894233 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8737e681-5d6a-4bc7-a280-c05bf4ae21ea tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:55.894233 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8737e681-5d6a-4bc7-a280-c05bf4ae21ea tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:55.902611 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6dddc720-0be5-4d78-9963-f2fc5ee19009 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:55.911151 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6dddc720-0be5-4d78-9963-f2fc5ee19009 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.172686 Aug 21 01:25:55.912905 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 76/148] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:55 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:56.061575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8737e681-5d6a-4bc7-a280-c05bf4ae21ea tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:56.069275 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8737e681-5d6a-4bc7-a280-c05bf4ae21ea tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.190404 Aug 21 01:25:56.069841 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 73/149] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:55 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:56.188596 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e8465da4-0964-4eb4-9bcc-3e25f5945167 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:56.189879 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e8465da4-0964-4eb4-9bcc-3e25f5945167 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:56.196020 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e8465da4-0964-4eb4-9bcc-3e25f5945167 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:56.196797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e8465da4-0964-4eb4-9bcc-3e25f5945167 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:56.197240 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e8465da4-0964-4eb4-9bcc-3e25f5945167 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:56.198144 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e8465da4-0964-4eb4-9bcc-3e25f5945167 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:56.198602 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e8465da4-0964-4eb4-9bcc-3e25f5945167 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:56.199027 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e8465da4-0964-4eb4-9bcc-3e25f5945167 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:56.361005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2759f75a-a485-4765-8b53-c13499848a5f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:56.361669 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2759f75a-a485-4765-8b53-c13499848a5f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:56.368214 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2759f75a-a485-4765-8b53-c13499848a5f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:56.368918 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2759f75a-a485-4765-8b53-c13499848a5f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:56.369354 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2759f75a-a485-4765-8b53-c13499848a5f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:56.370052 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2759f75a-a485-4765-8b53-c13499848a5f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:56.370615 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2759f75a-a485-4765-8b53-c13499848a5f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:56.370910 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2759f75a-a485-4765-8b53-c13499848a5f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:56.374998 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e8465da4-0964-4eb4-9bcc-3e25f5945167 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:56.387430 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e8465da4-0964-4eb4-9bcc-3e25f5945167 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:56.391498 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e8465da4-0964-4eb4-9bcc-3e25f5945167 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:56.391498 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e8465da4-0964-4eb4-9bcc-3e25f5945167 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.205664 Aug 21 01:25:56.391498 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 77/150] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:56 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:56.550494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2759f75a-a485-4765-8b53-c13499848a5f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:56.550494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2759f75a-a485-4765-8b53-c13499848a5f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.188419 Aug 21 01:25:56.550494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 74/151] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:56 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:56.932649 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5a163754-aaa5-4c80-9069-ec272bbd7fb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:56.933254 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5a163754-aaa5-4c80-9069-ec272bbd7fb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:56.940093 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a163754-aaa5-4c80-9069-ec272bbd7fb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:56.940690 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a163754-aaa5-4c80-9069-ec272bbd7fb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:56.941957 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a163754-aaa5-4c80-9069-ec272bbd7fb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:56.942744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a163754-aaa5-4c80-9069-ec272bbd7fb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:56.943198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a163754-aaa5-4c80-9069-ec272bbd7fb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:56.943623 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a163754-aaa5-4c80-9069-ec272bbd7fb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:57.094571 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2d74a7a2-0c21-4375-a540-9f435702cbc6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:57.094571 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2d74a7a2-0c21-4375-a540-9f435702cbc6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:57.104821 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2d74a7a2-0c21-4375-a540-9f435702cbc6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:57.105719 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2d74a7a2-0c21-4375-a540-9f435702cbc6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:57.106965 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2d74a7a2-0c21-4375-a540-9f435702cbc6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:57.109221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2d74a7a2-0c21-4375-a540-9f435702cbc6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:57.109914 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2d74a7a2-0c21-4375-a540-9f435702cbc6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:57.110549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2d74a7a2-0c21-4375-a540-9f435702cbc6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:57.178910 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5a163754-aaa5-4c80-9069-ec272bbd7fb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:57.187236 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5a163754-aaa5-4c80-9069-ec272bbd7fb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.258441 Aug 21 01:25:57.187826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 78/152] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:56 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:57.204091 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-4f9e571e-4afd-4326-bed8-ec37725b5a81 req-c535f2c1-a235-423d-946e-f1c94c767c4a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "031af209-ae76-47b8-8481-88ec4053affa", "name": "network-vif-plugged", "status": "completed", "tag": "667f35ba-e131-4619-82ff-965a65cd85fd"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:57.216402 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-4f9e571e-4afd-4326-bed8-ec37725b5a81 req-c535f2c1-a235-423d-946e-f1c94c767c4a service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:57.216950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-4f9e571e-4afd-4326-bed8-ec37725b5a81 req-c535f2c1-a235-423d-946e-f1c94c767c4a service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:57.217387 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-4f9e571e-4afd-4326-bed8-ec37725b5a81 req-c535f2c1-a235-423d-946e-f1c94c767c4a service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:57.232344 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-4f9e571e-4afd-4326-bed8-ec37725b5a81 req-c535f2c1-a235-423d-946e-f1c94c767c4a service nova] Creating event network-vif-plugged:667f35ba-e131-4619-82ff-965a65cd85fd for instance 031af209-ae76-47b8-8481-88ec4053affa on hv-cinder-maste Aug 21 01:25:57.239106 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-4f9e571e-4afd-4326-bed8-ec37725b5a81 req-c535f2c1-a235-423d-946e-f1c94c767c4a service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038108 Aug 21 01:25:57.239711 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 79/153] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:25:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:25:57.317167 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2d74a7a2-0c21-4375-a540-9f435702cbc6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:57.330559 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2d74a7a2-0c21-4375-a540-9f435702cbc6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.239600 Aug 21 01:25:57.334740 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 75/154] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:57 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:57.423246 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-fffd4112-6113-4762-a4cb-0fa8eff335c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:57.423246 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-fffd4112-6113-4762-a4cb-0fa8eff335c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:57.430465 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fffd4112-6113-4762-a4cb-0fa8eff335c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:57.431043 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fffd4112-6113-4762-a4cb-0fa8eff335c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:57.431556 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fffd4112-6113-4762-a4cb-0fa8eff335c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:57.432410 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fffd4112-6113-4762-a4cb-0fa8eff335c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:57.432949 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fffd4112-6113-4762-a4cb-0fa8eff335c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:57.433570 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fffd4112-6113-4762-a4cb-0fa8eff335c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:57.571004 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-026b086c-eb48-4bfb-87dc-c1f31586ae4d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:57.571715 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-026b086c-eb48-4bfb-87dc-c1f31586ae4d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:57.581917 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-026b086c-eb48-4bfb-87dc-c1f31586ae4d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:57.585496 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-026b086c-eb48-4bfb-87dc-c1f31586ae4d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:57.586047 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-026b086c-eb48-4bfb-87dc-c1f31586ae4d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:57.586977 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-026b086c-eb48-4bfb-87dc-c1f31586ae4d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:57.587444 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-026b086c-eb48-4bfb-87dc-c1f31586ae4d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:57.587878 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-026b086c-eb48-4bfb-87dc-c1f31586ae4d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:57.810380 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-fffd4112-6113-4762-a4cb-0fa8eff335c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:57.835059 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-fffd4112-6113-4762-a4cb-0fa8eff335c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:57.841966 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-fffd4112-6113-4762-a4cb-0fa8eff335c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:57.843123 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-fffd4112-6113-4762-a4cb-0fa8eff335c0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.428069 Aug 21 01:25:57.846544 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 80/155] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:57 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:57.957305 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-026b086c-eb48-4bfb-87dc-c1f31586ae4d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:57.967231 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-026b086c-eb48-4bfb-87dc-c1f31586ae4d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.401008 Aug 21 01:25:57.969782 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 76/156] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:57 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:58.208502 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-974ee2dd-c047-4f0a-a651-d2a971a1512d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:58.209275 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-974ee2dd-c047-4f0a-a651-d2a971a1512d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:58.215040 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-974ee2dd-c047-4f0a-a651-d2a971a1512d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:58.215556 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-974ee2dd-c047-4f0a-a651-d2a971a1512d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:58.216023 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-974ee2dd-c047-4f0a-a651-d2a971a1512d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:58.216803 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-974ee2dd-c047-4f0a-a651-d2a971a1512d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:58.217250 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-974ee2dd-c047-4f0a-a651-d2a971a1512d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:58.217672 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-974ee2dd-c047-4f0a-a651-d2a971a1512d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:58.353801 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f0076333-658e-4610-9b09-b6f9679ef6f0 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:58.354441 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f0076333-658e-4610-9b09-b6f9679ef6f0 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:58.360006 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-974ee2dd-c047-4f0a-a651-d2a971a1512d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:58.364408 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f0076333-658e-4610-9b09-b6f9679ef6f0 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:58.364408 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f0076333-658e-4610-9b09-b6f9679ef6f0 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:58.364408 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f0076333-658e-4610-9b09-b6f9679ef6f0 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:58.364408 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f0076333-658e-4610-9b09-b6f9679ef6f0 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:58.364856 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f0076333-658e-4610-9b09-b6f9679ef6f0 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:58.364856 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f0076333-658e-4610-9b09-b6f9679ef6f0 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:58.369938 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-974ee2dd-c047-4f0a-a651-d2a971a1512d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.165805 Aug 21 01:25:58.370750 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 81/157] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:58 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:58.514637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f0076333-658e-4610-9b09-b6f9679ef6f0 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:58.525593 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f0076333-658e-4610-9b09-b6f9679ef6f0 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.176067 Aug 21 01:25:58.526703 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 77/158] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:58 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:58.870928 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d3da62e3-dddc-4623-adc0-3c39d5a7d33e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:58.871660 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d3da62e3-dddc-4623-adc0-3c39d5a7d33e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:58.877773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d3da62e3-dddc-4623-adc0-3c39d5a7d33e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:58.878276 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d3da62e3-dddc-4623-adc0-3c39d5a7d33e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:58.878917 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d3da62e3-dddc-4623-adc0-3c39d5a7d33e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:58.879610 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d3da62e3-dddc-4623-adc0-3c39d5a7d33e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:58.880060 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d3da62e3-dddc-4623-adc0-3c39d5a7d33e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:58.880629 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d3da62e3-dddc-4623-adc0-3c39d5a7d33e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:58.990087 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bfd654d3-40f3-4dd5-9f3e-d66d435d93c2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:58.991274 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bfd654d3-40f3-4dd5-9f3e-d66d435d93c2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:59.003030 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bfd654d3-40f3-4dd5-9f3e-d66d435d93c2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:59.004414 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bfd654d3-40f3-4dd5-9f3e-d66d435d93c2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:59.005612 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bfd654d3-40f3-4dd5-9f3e-d66d435d93c2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:59.007503 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bfd654d3-40f3-4dd5-9f3e-d66d435d93c2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:59.009146 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bfd654d3-40f3-4dd5-9f3e-d66d435d93c2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:59.010206 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bfd654d3-40f3-4dd5-9f3e-d66d435d93c2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:59.035087 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d3da62e3-dddc-4623-adc0-3c39d5a7d33e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:59.043818 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d3da62e3-dddc-4623-adc0-3c39d5a7d33e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:59.045560 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d3da62e3-dddc-4623-adc0-3c39d5a7d33e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:59.047174 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d3da62e3-dddc-4623-adc0-3c39d5a7d33e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.180694 Aug 21 01:25:59.047814 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 82/159] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:25:58 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:25:59.172522 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-bfd654d3-40f3-4dd5-9f3e-d66d435d93c2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:59.180482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bfd654d3-40f3-4dd5-9f3e-d66d435d93c2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.196423 Aug 21 01:25:59.183838 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 78/160] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:58 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:25:59.256607 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-27e6b0ee-b663-48c6-9f9c-55ffc71cb790 req-e0917ebf-0f2f-4f95-aeea-abd4102457b9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "80c72a3e-79b5-48fa-9c5f-26a858b589de", "name": "network-vif-plugged", "status": "completed", "tag": "f7aa7447-c670-4701-a6ac-18f99415ce92"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:25:59.266412 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-27e6b0ee-b663-48c6-9f9c-55ffc71cb790 req-e0917ebf-0f2f-4f95-aeea-abd4102457b9 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:59.266887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-27e6b0ee-b663-48c6-9f9c-55ffc71cb790 req-e0917ebf-0f2f-4f95-aeea-abd4102457b9 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:59.267363 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-27e6b0ee-b663-48c6-9f9c-55ffc71cb790 req-e0917ebf-0f2f-4f95-aeea-abd4102457b9 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:59.280012 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-27e6b0ee-b663-48c6-9f9c-55ffc71cb790 req-e0917ebf-0f2f-4f95-aeea-abd4102457b9 service nova] Creating event network-vif-plugged:f7aa7447-c670-4701-a6ac-18f99415ce92 for instance 80c72a3e-79b5-48fa-9c5f-26a858b589de on hv-cinder-maste Aug 21 01:25:59.286111 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-27e6b0ee-b663-48c6-9f9c-55ffc71cb790 req-e0917ebf-0f2f-4f95-aeea-abd4102457b9 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033631 Aug 21 01:25:59.291272 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 83/161] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:25:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:25:59.394699 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9ae3bc43-109d-4a28-b646-37eed3053270 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:59.395385 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9ae3bc43-109d-4a28-b646-37eed3053270 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:59.401645 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ae3bc43-109d-4a28-b646-37eed3053270 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:59.402297 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ae3bc43-109d-4a28-b646-37eed3053270 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:59.406135 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ae3bc43-109d-4a28-b646-37eed3053270 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:59.406135 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ae3bc43-109d-4a28-b646-37eed3053270 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:59.406135 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ae3bc43-109d-4a28-b646-37eed3053270 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:59.406135 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ae3bc43-109d-4a28-b646-37eed3053270 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:59.541937 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9ae3bc43-109d-4a28-b646-37eed3053270 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:59.547804 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5a140db8-08c5-48d7-a803-9066f469700d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:25:59.548776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5a140db8-08c5-48d7-a803-9066f469700d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:25:59.549941 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9ae3bc43-109d-4a28-b646-37eed3053270 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.159694 Aug 21 01:25:59.550750 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 79/162] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:59 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:25:59.555330 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a140db8-08c5-48d7-a803-9066f469700d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:59.555888 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a140db8-08c5-48d7-a803-9066f469700d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:59.556429 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a140db8-08c5-48d7-a803-9066f469700d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:59.557328 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a140db8-08c5-48d7-a803-9066f469700d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:25:59.557851 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a140db8-08c5-48d7-a803-9066f469700d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:25:59.558358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a140db8-08c5-48d7-a803-9066f469700d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:25:59.772684 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5a140db8-08c5-48d7-a803-9066f469700d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:25:59.780727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5a140db8-08c5-48d7-a803-9066f469700d tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.237180 Aug 21 01:25:59.781305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 84/163] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:25:59 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:00.070274 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6fcf5224-7b64-43f9-98b5-0cdfffc59571 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:00.070870 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6fcf5224-7b64-43f9-98b5-0cdfffc59571 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:00.072849 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=97 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:26:00.078032 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6fcf5224-7b64-43f9-98b5-0cdfffc59571 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:00.078518 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6fcf5224-7b64-43f9-98b5-0cdfffc59571 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:00.078949 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6fcf5224-7b64-43f9-98b5-0cdfffc59571 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:00.079637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6fcf5224-7b64-43f9-98b5-0cdfffc59571 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:00.080084 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6fcf5224-7b64-43f9-98b5-0cdfffc59571 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:00.080528 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6fcf5224-7b64-43f9-98b5-0cdfffc59571 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:00.204624 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-57555c6c-2602-48dd-ab47-445cec0935a2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:00.205263 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-57555c6c-2602-48dd-ab47-445cec0935a2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:00.214043 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-57555c6c-2602-48dd-ab47-445cec0935a2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:00.214548 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-57555c6c-2602-48dd-ab47-445cec0935a2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:00.214986 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-57555c6c-2602-48dd-ab47-445cec0935a2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:00.215685 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-57555c6c-2602-48dd-ab47-445cec0935a2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:00.216347 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-57555c6c-2602-48dd-ab47-445cec0935a2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:00.216782 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-57555c6c-2602-48dd-ab47-445cec0935a2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:00.233697 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6fcf5224-7b64-43f9-98b5-0cdfffc59571 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:00.240981 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=100 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:26:00.249311 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6fcf5224-7b64-43f9-98b5-0cdfffc59571 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:00.251690 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6fcf5224-7b64-43f9-98b5-0cdfffc59571 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:00.253518 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6fcf5224-7b64-43f9-98b5-0cdfffc59571 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.188109 Aug 21 01:26:00.254253 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 80/164] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:26:00 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:26:00.381874 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-57555c6c-2602-48dd-ab47-445cec0935a2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:00.383362 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-57555c6c-2602-48dd-ab47-445cec0935a2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.184380 Aug 21 01:26:00.384159 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 85/165] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:00 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:26:00.571385 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-79cf577d-4e56-47b3-998a-219f4354e78c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:00.578231 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-79cf577d-4e56-47b3-998a-219f4354e78c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:00.578231 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-79cf577d-4e56-47b3-998a-219f4354e78c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:00.578829 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-79cf577d-4e56-47b3-998a-219f4354e78c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:00.579261 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-79cf577d-4e56-47b3-998a-219f4354e78c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:00.579941 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-79cf577d-4e56-47b3-998a-219f4354e78c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:00.580424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-79cf577d-4e56-47b3-998a-219f4354e78c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:00.580841 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-79cf577d-4e56-47b3-998a-219f4354e78c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:00.811317 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f6d70963-2516-47f2-8067-bf4184e0c496 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:00.812174 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f6d70963-2516-47f2-8067-bf4184e0c496 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:00.822461 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f6d70963-2516-47f2-8067-bf4184e0c496 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:00.822965 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f6d70963-2516-47f2-8067-bf4184e0c496 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:00.823404 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f6d70963-2516-47f2-8067-bf4184e0c496 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:00.825233 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f6d70963-2516-47f2-8067-bf4184e0c496 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:00.826328 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f6d70963-2516-47f2-8067-bf4184e0c496 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:00.829422 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f6d70963-2516-47f2-8067-bf4184e0c496 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:00.833370 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-79cf577d-4e56-47b3-998a-219f4354e78c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:00.843542 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-79cf577d-4e56-47b3-998a-219f4354e78c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.276340 Aug 21 01:26:00.849547 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 81/166] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:00 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:01.285449 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-26177ccf-7e01-496f-8450-e8910e4acb88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:01.286365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-26177ccf-7e01-496f-8450-e8910e4acb88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:01.292888 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-26177ccf-7e01-496f-8450-e8910e4acb88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:01.293813 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-26177ccf-7e01-496f-8450-e8910e4acb88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:01.294158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-26177ccf-7e01-496f-8450-e8910e4acb88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:01.295106 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-26177ccf-7e01-496f-8450-e8910e4acb88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:01.295682 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-26177ccf-7e01-496f-8450-e8910e4acb88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:01.296481 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-26177ccf-7e01-496f-8450-e8910e4acb88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:01.315844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f6d70963-2516-47f2-8067-bf4184e0c496 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:01.325336 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f6d70963-2516-47f2-8067-bf4184e0c496 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1315 microversion: 2.1 time: 0.526909 Aug 21 01:26:01.328061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 86/167] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:00 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1315 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:01.353825 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-a35433f7-df55-4881-8ecb-e49b37709718 req-91a34757-dfd1-4380-a420-2c53d07fb8c5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c266a09a-3f81-4111-a987-b1f4be65cdc4", "name": "network-vif-plugged", "status": "completed", "tag": "8d6fe117-8bd6-47dc-9dd9-1b92667e8ba7"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:26:01.364833 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-a35433f7-df55-4881-8ecb-e49b37709718 req-91a34757-dfd1-4380-a420-2c53d07fb8c5 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:01.365639 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-a35433f7-df55-4881-8ecb-e49b37709718 req-91a34757-dfd1-4380-a420-2c53d07fb8c5 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:01.366076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-a35433f7-df55-4881-8ecb-e49b37709718 req-91a34757-dfd1-4380-a420-2c53d07fb8c5 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:01.379364 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-a35433f7-df55-4881-8ecb-e49b37709718 req-91a34757-dfd1-4380-a420-2c53d07fb8c5 service nova] Creating event network-vif-plugged:8d6fe117-8bd6-47dc-9dd9-1b92667e8ba7 for instance c266a09a-3f81-4111-a987-b1f4be65cdc4 on hv-cinder-maste Aug 21 01:26:01.385677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-a35433f7-df55-4881-8ecb-e49b37709718 req-91a34757-dfd1-4380-a420-2c53d07fb8c5 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055402 Aug 21 01:26:01.386882 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 87/168] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:26:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:26:01.412655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e3b706f8-00c1-4232-829c-fcce1c835beb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:01.414777 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e3b706f8-00c1-4232-829c-fcce1c835beb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:01.427608 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3b706f8-00c1-4232-829c-fcce1c835beb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:01.429088 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3b706f8-00c1-4232-829c-fcce1c835beb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:01.430067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3b706f8-00c1-4232-829c-fcce1c835beb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:01.430342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3b706f8-00c1-4232-829c-fcce1c835beb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:01.431288 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3b706f8-00c1-4232-829c-fcce1c835beb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:01.431863 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3b706f8-00c1-4232-829c-fcce1c835beb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:01.525925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-26177ccf-7e01-496f-8450-e8910e4acb88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:01.535604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-26177ccf-7e01-496f-8450-e8910e4acb88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:01.537153 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-26177ccf-7e01-496f-8450-e8910e4acb88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:01.539143 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-26177ccf-7e01-496f-8450-e8910e4acb88 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.257554 Aug 21 01:26:01.539143 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 82/169] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:26:01 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:26:01.588971 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e3b706f8-00c1-4232-829c-fcce1c835beb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:01.597152 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e3b706f8-00c1-4232-829c-fcce1c835beb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.195537 Aug 21 01:26:01.597920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 88/170] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:01 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:01.873616 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ddaea731-410c-4f4c-9e93-33f506159927 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:01.874221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ddaea731-410c-4f4c-9e93-33f506159927 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:01.881281 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ddaea731-410c-4f4c-9e93-33f506159927 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:01.881786 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ddaea731-410c-4f4c-9e93-33f506159927 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:01.882202 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ddaea731-410c-4f4c-9e93-33f506159927 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:01.882895 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ddaea731-410c-4f4c-9e93-33f506159927 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:01.883344 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ddaea731-410c-4f4c-9e93-33f506159927 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:01.883783 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ddaea731-410c-4f4c-9e93-33f506159927 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:02.073512 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ddaea731-410c-4f4c-9e93-33f506159927 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:02.082064 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ddaea731-410c-4f4c-9e93-33f506159927 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1322 microversion: 2.1 time: 0.212196 Aug 21 01:26:02.092309 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 83/171] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:01 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1322 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:02.354094 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-1e2b99b4-bc9c-42a7-bc48-43d50e0929c5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:02.354947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-1e2b99b4-bc9c-42a7-bc48-43d50e0929c5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:02.362575 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1e2b99b4-bc9c-42a7-bc48-43d50e0929c5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:02.363158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1e2b99b4-bc9c-42a7-bc48-43d50e0929c5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:02.363608 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1e2b99b4-bc9c-42a7-bc48-43d50e0929c5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:02.364488 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1e2b99b4-bc9c-42a7-bc48-43d50e0929c5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:02.364950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1e2b99b4-bc9c-42a7-bc48-43d50e0929c5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:02.365411 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1e2b99b4-bc9c-42a7-bc48-43d50e0929c5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:02.370887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=100 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:26:02.505014 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-1e2b99b4-bc9c-42a7-bc48-43d50e0929c5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:02.512776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-1e2b99b4-bc9c-42a7-bc48-43d50e0929c5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1502 microversion: 2.1 time: 0.167448 Aug 21 01:26:02.513350 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 89/172] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:02 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1502 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:02.565714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3dbcc7c5-a2e1-4d1f-a003-2311f11886a6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:02.566575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3dbcc7c5-a2e1-4d1f-a003-2311f11886a6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:02.573357 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3dbcc7c5-a2e1-4d1f-a003-2311f11886a6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:02.573880 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3dbcc7c5-a2e1-4d1f-a003-2311f11886a6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:02.574313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3dbcc7c5-a2e1-4d1f-a003-2311f11886a6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:02.575203 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3dbcc7c5-a2e1-4d1f-a003-2311f11886a6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:02.575654 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3dbcc7c5-a2e1-4d1f-a003-2311f11886a6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:02.576078 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3dbcc7c5-a2e1-4d1f-a003-2311f11886a6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:02.630776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3221772a-9ae8-47f5-bd97-62a00fe8f736 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:02.641559 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3221772a-9ae8-47f5-bd97-62a00fe8f736 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:02.665826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3221772a-9ae8-47f5-bd97-62a00fe8f736 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:02.665826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3221772a-9ae8-47f5-bd97-62a00fe8f736 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:02.666063 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3221772a-9ae8-47f5-bd97-62a00fe8f736 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:02.666641 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3221772a-9ae8-47f5-bd97-62a00fe8f736 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:02.667097 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3221772a-9ae8-47f5-bd97-62a00fe8f736 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:02.669983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3221772a-9ae8-47f5-bd97-62a00fe8f736 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:02.959552 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3dbcc7c5-a2e1-4d1f-a003-2311f11886a6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:02.983820 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3dbcc7c5-a2e1-4d1f-a003-2311f11886a6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:02.990458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3dbcc7c5-a2e1-4d1f-a003-2311f11886a6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:02.991779 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3dbcc7c5-a2e1-4d1f-a003-2311f11886a6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.430446 Aug 21 01:26:02.992438 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 84/173] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:26:02 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:26:03.016196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3221772a-9ae8-47f5-bd97-62a00fe8f736 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:03.026564 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3221772a-9ae8-47f5-bd97-62a00fe8f736 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.412219 Aug 21 01:26:03.027342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 90/174] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:02 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:03.103668 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-db006a86-5d20-41cc-a2b4-1d5f4013ddc1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:03.105308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-db006a86-5d20-41cc-a2b4-1d5f4013ddc1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:03.113616 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db006a86-5d20-41cc-a2b4-1d5f4013ddc1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:03.115828 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db006a86-5d20-41cc-a2b4-1d5f4013ddc1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:03.116463 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db006a86-5d20-41cc-a2b4-1d5f4013ddc1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:03.121855 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db006a86-5d20-41cc-a2b4-1d5f4013ddc1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:03.121855 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db006a86-5d20-41cc-a2b4-1d5f4013ddc1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:03.121855 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db006a86-5d20-41cc-a2b4-1d5f4013ddc1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:03.319597 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-db006a86-5d20-41cc-a2b4-1d5f4013ddc1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:03.327933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-db006a86-5d20-41cc-a2b4-1d5f4013ddc1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1515 microversion: 2.1 time: 0.228826 Aug 21 01:26:03.335049 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 85/175] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:03 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1515 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:03.351146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c483afd9-f156-43d7-ae94-c7bc51ac8f91 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:26:03.354018 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c483afd9-f156-43d7-ae94-c7bc51ac8f91 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:03.363610 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c483afd9-f156-43d7-ae94-c7bc51ac8f91 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:03.364332 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c483afd9-f156-43d7-ae94-c7bc51ac8f91 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:03.364957 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c483afd9-f156-43d7-ae94-c7bc51ac8f91 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:03.365272 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c483afd9-f156-43d7-ae94-c7bc51ac8f91 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:03.365755 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c483afd9-f156-43d7-ae94-c7bc51ac8f91 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:03.366175 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c483afd9-f156-43d7-ae94-c7bc51ac8f91 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:03.407594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-161e54f9-b4ce-4787-8e15-3db827f53840 req-5cca9df5-840f-4737-ba54-ea026a754d4d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1", "name": "network-vif-plugged", "status": "completed", "tag": "9a09b4f3-32e5-4362-82ef-ab73b634b1bf"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:26:03.419275 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-161e54f9-b4ce-4787-8e15-3db827f53840 req-5cca9df5-840f-4737-ba54-ea026a754d4d service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:03.420244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-161e54f9-b4ce-4787-8e15-3db827f53840 req-5cca9df5-840f-4737-ba54-ea026a754d4d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:03.420834 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-161e54f9-b4ce-4787-8e15-3db827f53840 req-5cca9df5-840f-4737-ba54-ea026a754d4d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:03.434797 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-161e54f9-b4ce-4787-8e15-3db827f53840 req-5cca9df5-840f-4737-ba54-ea026a754d4d service nova] Creating event network-vif-plugged:9a09b4f3-32e5-4362-82ef-ab73b634b1bf for instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 on hv-cinder-maste Aug 21 01:26:03.440909 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-c483afd9-f156-43d7-ae94-c7bc51ac8f91 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lazy-loading 'flavor' on Instance uuid 80c72a3e-79b5-48fa-9c5f-26a858b589de {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:26:03.442364 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-161e54f9-b4ce-4787-8e15-3db827f53840 req-5cca9df5-840f-4737-ba54-ea026a754d4d service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038479 Aug 21 01:26:03.443045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 86/176] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:26:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:26:04.015674 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-11a89eb6-063c-4b1d-ad9f-e74cd11f5528 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:04.016454 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-11a89eb6-063c-4b1d-ad9f-e74cd11f5528 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:04.023136 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-11a89eb6-063c-4b1d-ad9f-e74cd11f5528 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:04.023616 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-11a89eb6-063c-4b1d-ad9f-e74cd11f5528 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:04.024045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-11a89eb6-063c-4b1d-ad9f-e74cd11f5528 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:04.024756 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-11a89eb6-063c-4b1d-ad9f-e74cd11f5528 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:04.025200 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-11a89eb6-063c-4b1d-ad9f-e74cd11f5528 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:04.025613 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-11a89eb6-063c-4b1d-ad9f-e74cd11f5528 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:04.202911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-11a89eb6-063c-4b1d-ad9f-e74cd11f5528 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:04.212062 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-11a89eb6-063c-4b1d-ad9f-e74cd11f5528 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:04.214412 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-11a89eb6-063c-4b1d-ad9f-e74cd11f5528 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:04.214847 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-11a89eb6-063c-4b1d-ad9f-e74cd11f5528 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.203432 Aug 21 01:26:04.215417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 87/177] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:26:04 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:26:04.223755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-06b766e3-fbf6-4f02-95ba-a1af76d791d0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:04.224625 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-06b766e3-fbf6-4f02-95ba-a1af76d791d0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:04.231425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06b766e3-fbf6-4f02-95ba-a1af76d791d0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:04.232065 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06b766e3-fbf6-4f02-95ba-a1af76d791d0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:04.232569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06b766e3-fbf6-4f02-95ba-a1af76d791d0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:04.233322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06b766e3-fbf6-4f02-95ba-a1af76d791d0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:04.233937 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06b766e3-fbf6-4f02-95ba-a1af76d791d0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:04.234250 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06b766e3-fbf6-4f02-95ba-a1af76d791d0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:04.272251 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c483afd9-f156-43d7-ae94-c7bc51ac8f91 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "POST /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de/action" status: 200 len: 32 microversion: 2.1 time: 0.927045 Aug 21 01:26:04.277650 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 91/178] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:26:03 2022] POST /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de/action => generated 32 bytes in 929 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 21 01:26:04.287561 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9c599a1c-3a1f-4864-a9d2-bee8ade3de1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:04.288604 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9c599a1c-3a1f-4864-a9d2-bee8ade3de1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:04.302985 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9c599a1c-3a1f-4864-a9d2-bee8ade3de1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:04.302985 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9c599a1c-3a1f-4864-a9d2-bee8ade3de1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:04.303254 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9c599a1c-3a1f-4864-a9d2-bee8ade3de1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:04.303920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9c599a1c-3a1f-4864-a9d2-bee8ade3de1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:04.304422 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9c599a1c-3a1f-4864-a9d2-bee8ade3de1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:04.305709 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9c599a1c-3a1f-4864-a9d2-bee8ade3de1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:04.462599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-06b766e3-fbf6-4f02-95ba-a1af76d791d0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:04.472746 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-06b766e3-fbf6-4f02-95ba-a1af76d791d0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1363 microversion: 2.1 time: 0.256106 Aug 21 01:26:04.473529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 88/179] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:04 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1363 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:04.544433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9c599a1c-3a1f-4864-a9d2-bee8ade3de1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:04.552780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9c599a1c-3a1f-4864-a9d2-bee8ade3de1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.271143 Aug 21 01:26:04.553382 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 92/180] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:04 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:04.681090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-66972871-c709-4cda-8b6c-8de71f84ab0b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b783abf1-6e46-4d38-8936-8ca316cb6e58"}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:26:04.684092 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-66972871-c709-4cda-8b6c-8de71f84ab0b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:04.692270 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-66972871-c709-4cda-8b6c-8de71f84ab0b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:04.692826 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-66972871-c709-4cda-8b6c-8de71f84ab0b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:04.693359 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-66972871-c709-4cda-8b6c-8de71f84ab0b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:04.694189 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-66972871-c709-4cda-8b6c-8de71f84ab0b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:04.694959 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-66972871-c709-4cda-8b6c-8de71f84ab0b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:04.695795 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-66972871-c709-4cda-8b6c-8de71f84ab0b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:05.239066 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-03293f90-37c3-4af5-bde3-18478ab285d6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:05.239907 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-03293f90-37c3-4af5-bde3-18478ab285d6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:05.246192 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-03293f90-37c3-4af5-bde3-18478ab285d6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:05.246685 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-03293f90-37c3-4af5-bde3-18478ab285d6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:05.247120 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-03293f90-37c3-4af5-bde3-18478ab285d6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:05.248067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-03293f90-37c3-4af5-bde3-18478ab285d6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:05.248279 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-03293f90-37c3-4af5-bde3-18478ab285d6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:05.248706 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-03293f90-37c3-4af5-bde3-18478ab285d6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:05.402321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-03293f90-37c3-4af5-bde3-18478ab285d6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:05.411304 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-03293f90-37c3-4af5-bde3-18478ab285d6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:05.413336 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-03293f90-37c3-4af5-bde3-18478ab285d6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:05.414505 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-03293f90-37c3-4af5-bde3-18478ab285d6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.179501 Aug 21 01:26:05.415213 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 93/181] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:26:05 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:26:05.504886 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2189fdf9-836c-4c59-8142-769cd09e6b1c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:05.505848 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-2189fdf9-836c-4c59-8142-769cd09e6b1c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:05.514837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2189fdf9-836c-4c59-8142-769cd09e6b1c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:05.516398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2189fdf9-836c-4c59-8142-769cd09e6b1c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:05.523210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2189fdf9-836c-4c59-8142-769cd09e6b1c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:05.524166 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2189fdf9-836c-4c59-8142-769cd09e6b1c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:05.524783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2189fdf9-836c-4c59-8142-769cd09e6b1c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:05.525417 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2189fdf9-836c-4c59-8142-769cd09e6b1c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:05.781465 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2189fdf9-836c-4c59-8142-769cd09e6b1c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:05.790087 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2189fdf9-836c-4c59-8142-769cd09e6b1c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1550 microversion: 2.1 time: 0.292649 Aug 21 01:26:05.790678 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 94/182] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:05 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1550 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:26:05.795357 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6baa58f1-81c2-4a01-85f3-eaa685789d4b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:05.795961 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6baa58f1-81c2-4a01-85f3-eaa685789d4b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:05.798988 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=100 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:26:05.812679 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6baa58f1-81c2-4a01-85f3-eaa685789d4b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:05.821767 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6baa58f1-81c2-4a01-85f3-eaa685789d4b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:05.821767 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6baa58f1-81c2-4a01-85f3-eaa685789d4b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:05.821767 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6baa58f1-81c2-4a01-85f3-eaa685789d4b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:05.821767 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6baa58f1-81c2-4a01-85f3-eaa685789d4b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:05.822380 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6baa58f1-81c2-4a01-85f3-eaa685789d4b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:06.009564 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6baa58f1-81c2-4a01-85f3-eaa685789d4b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:06.017659 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6baa58f1-81c2-4a01-85f3-eaa685789d4b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.225797 Aug 21 01:26:06.018886 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 95/183] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:05 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:06.024247 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b0b126c1-ce78-4010-be07-40e2f22cb94b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:06.024764 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b0b126c1-ce78-4010-be07-40e2f22cb94b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:06.032700 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b0b126c1-ce78-4010-be07-40e2f22cb94b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:06.033099 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b0b126c1-ce78-4010-be07-40e2f22cb94b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:06.033614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b0b126c1-ce78-4010-be07-40e2f22cb94b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:06.034875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b0b126c1-ce78-4010-be07-40e2f22cb94b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:06.035329 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b0b126c1-ce78-4010-be07-40e2f22cb94b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:06.035929 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b0b126c1-ce78-4010-be07-40e2f22cb94b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:06.127183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b0b126c1-ce78-4010-be07-40e2f22cb94b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.107114 Aug 21 01:26:06.127813 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 96/184] 10.222.0.47 () {58 vars in 1275 bytes} [Sun Aug 21 01:26:06 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/os-interface => generated 285 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:26:06.438015 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-41f45c61-0cfb-4539-8318-e19a8ebaebc7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:06.438830 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-41f45c61-0cfb-4539-8318-e19a8ebaebc7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:06.447670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-41f45c61-0cfb-4539-8318-e19a8ebaebc7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:06.448483 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-41f45c61-0cfb-4539-8318-e19a8ebaebc7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:06.448928 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-41f45c61-0cfb-4539-8318-e19a8ebaebc7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:06.449793 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-41f45c61-0cfb-4539-8318-e19a8ebaebc7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:06.450260 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-41f45c61-0cfb-4539-8318-e19a8ebaebc7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:06.450686 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-41f45c61-0cfb-4539-8318-e19a8ebaebc7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:06.781193 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-41f45c61-0cfb-4539-8318-e19a8ebaebc7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:06.790222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-66972871-c709-4cda-8b6c-8de71f84ab0b tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "POST /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.115675 Aug 21 01:26:06.800085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 89/185] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:26:04 2022] POST /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa/os-volume_attachments => generated 194 bytes in 2128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:26:06.818294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-41f45c61-0cfb-4539-8318-e19a8ebaebc7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:06.820066 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-41f45c61-0cfb-4539-8318-e19a8ebaebc7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:06.821287 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-41f45c61-0cfb-4539-8318-e19a8ebaebc7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1406 microversion: 2.20 time: 0.388278 Aug 21 01:26:06.836128 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 97/186] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:26:06 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1406 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:26:07.050393 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-06db6e1a-c3b4-4272-910c-4e8920c91dab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:07.051607 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-06db6e1a-c3b4-4272-910c-4e8920c91dab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:07.059034 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06db6e1a-c3b4-4272-910c-4e8920c91dab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:07.059749 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06db6e1a-c3b4-4272-910c-4e8920c91dab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:07.060394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06db6e1a-c3b4-4272-910c-4e8920c91dab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:07.061355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06db6e1a-c3b4-4272-910c-4e8920c91dab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:07.061990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06db6e1a-c3b4-4272-910c-4e8920c91dab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:07.062402 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06db6e1a-c3b4-4272-910c-4e8920c91dab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:07.405320 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-06db6e1a-c3b4-4272-910c-4e8920c91dab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:07.414244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-06db6e1a-c3b4-4272-910c-4e8920c91dab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.369687 Aug 21 01:26:07.414911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 90/187] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:07 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:07.847388 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-bcc15aec-b766-42c8-85cd-bbf9f907da18 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:07.848806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-bcc15aec-b766-42c8-85cd-bbf9f907da18 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:07.856920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bcc15aec-b766-42c8-85cd-bbf9f907da18 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:07.857725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bcc15aec-b766-42c8-85cd-bbf9f907da18 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:07.858435 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bcc15aec-b766-42c8-85cd-bbf9f907da18 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:07.864584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bcc15aec-b766-42c8-85cd-bbf9f907da18 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:07.864584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bcc15aec-b766-42c8-85cd-bbf9f907da18 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:07.864584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bcc15aec-b766-42c8-85cd-bbf9f907da18 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:08.097096 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-c7205237-d935-49c0-9ee0-d3003af2ffa9 req-1b0fbf63-5e36-4cf6-a63e-892a309bd64d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c266a09a-3f81-4111-a987-b1f4be65cdc4", "tag": "8d6fe117-8bd6-47dc-9dd9-1b92667e8ba7"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:26:08.097651 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-bcc15aec-b766-42c8-85cd-bbf9f907da18 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:08.118647 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-bcc15aec-b766-42c8-85cd-bbf9f907da18 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:08.120433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-bcc15aec-b766-42c8-85cd-bbf9f907da18 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:08.122125 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-c7205237-d935-49c0-9ee0-d3003af2ffa9 req-1b0fbf63-5e36-4cf6-a63e-892a309bd64d service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:08.123511 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-bcc15aec-b766-42c8-85cd-bbf9f907da18 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1597 microversion: 2.20 time: 0.281765 Aug 21 01:26:08.124245 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-c7205237-d935-49c0-9ee0-d3003af2ffa9 req-1b0fbf63-5e36-4cf6-a63e-892a309bd64d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:08.125536 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-c7205237-d935-49c0-9ee0-d3003af2ffa9 req-1b0fbf63-5e36-4cf6-a63e-892a309bd64d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:08.125883 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 98/188] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:26:07 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1597 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:26:08.139240 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-c7205237-d935-49c0-9ee0-d3003af2ffa9 req-1b0fbf63-5e36-4cf6-a63e-892a309bd64d service nova] Creating event network-changed:8d6fe117-8bd6-47dc-9dd9-1b92667e8ba7 for instance c266a09a-3f81-4111-a987-b1f4be65cdc4 on hv-cinder-maste Aug 21 01:26:08.144850 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-c7205237-d935-49c0-9ee0-d3003af2ffa9 req-1b0fbf63-5e36-4cf6-a63e-892a309bd64d service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055987 Aug 21 01:26:08.149226 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 91/189] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:26:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:26:08.160693 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8a7a1eab-2841-46e2-bf55-aa1781f6c15a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:08.161185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8a7a1eab-2841-46e2-bf55-aa1781f6c15a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:08.175373 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8a7a1eab-2841-46e2-bf55-aa1781f6c15a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:08.176014 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8a7a1eab-2841-46e2-bf55-aa1781f6c15a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:08.176502 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8a7a1eab-2841-46e2-bf55-aa1781f6c15a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:08.177210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8a7a1eab-2841-46e2-bf55-aa1781f6c15a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:08.177658 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8a7a1eab-2841-46e2-bf55-aa1781f6c15a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:08.178715 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8a7a1eab-2841-46e2-bf55-aa1781f6c15a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:08.331008 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8a7a1eab-2841-46e2-bf55-aa1781f6c15a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.174203 Aug 21 01:26:08.331692 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 99/190] 10.222.0.47 () {60 vars in 1316 bytes} [Sun Aug 21 01:26:08 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-interface => generated 284 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:26:08.436935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c35f54cd-e3eb-4575-abf4-58d35579efed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:08.437659 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c35f54cd-e3eb-4575-abf4-58d35579efed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:08.454351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c35f54cd-e3eb-4575-abf4-58d35579efed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:08.456166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c35f54cd-e3eb-4575-abf4-58d35579efed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:08.460627 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c35f54cd-e3eb-4575-abf4-58d35579efed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:08.461599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c35f54cd-e3eb-4575-abf4-58d35579efed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:08.462247 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c35f54cd-e3eb-4575-abf4-58d35579efed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:08.465320 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c35f54cd-e3eb-4575-abf4-58d35579efed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:08.786449 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c35f54cd-e3eb-4575-abf4-58d35579efed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:08.801562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c35f54cd-e3eb-4575-abf4-58d35579efed tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.365199 Aug 21 01:26:08.802261 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 92/191] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:08 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:09.833596 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-939ffa04-e9eb-4dd6-af83-436beec2503e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:09.834209 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-939ffa04-e9eb-4dd6-af83-436beec2503e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:09.848679 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-939ffa04-e9eb-4dd6-af83-436beec2503e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:09.850729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-939ffa04-e9eb-4dd6-af83-436beec2503e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:09.851181 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-939ffa04-e9eb-4dd6-af83-436beec2503e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:09.853247 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-939ffa04-e9eb-4dd6-af83-436beec2503e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:09.853720 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-939ffa04-e9eb-4dd6-af83-436beec2503e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:09.854999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-939ffa04-e9eb-4dd6-af83-436beec2503e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:10.130826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-939ffa04-e9eb-4dd6-af83-436beec2503e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:10.141691 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-939ffa04-e9eb-4dd6-af83-436beec2503e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.314455 Aug 21 01:26:10.144253 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 100/192] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:09 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:10.178604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-5b811897-0bd2-4fac-abb7-150e2aeb95f7 req-f558d897-fbb7-49ec-8745-b286f688d035 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1", "tag": "9a09b4f3-32e5-4362-82ef-ab73b634b1bf"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:26:10.206348 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-5b811897-0bd2-4fac-abb7-150e2aeb95f7 req-f558d897-fbb7-49ec-8745-b286f688d035 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:10.207316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-5b811897-0bd2-4fac-abb7-150e2aeb95f7 req-f558d897-fbb7-49ec-8745-b286f688d035 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:10.208764 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-5b811897-0bd2-4fac-abb7-150e2aeb95f7 req-f558d897-fbb7-49ec-8745-b286f688d035 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:10.238267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-5b811897-0bd2-4fac-abb7-150e2aeb95f7 req-f558d897-fbb7-49ec-8745-b286f688d035 service nova] Creating event network-changed:9a09b4f3-32e5-4362-82ef-ab73b634b1bf for instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 on hv-cinder-maste Aug 21 01:26:10.251199 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-5b811897-0bd2-4fac-abb7-150e2aeb95f7 req-f558d897-fbb7-49ec-8745-b286f688d035 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086547 Aug 21 01:26:10.252361 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 93/193] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:26:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:26:11.165627 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6a458c91-9b24-42b2-a8a4-a2ab8f4fa0b0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:11.166127 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6a458c91-9b24-42b2-a8a4-a2ab8f4fa0b0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:11.172057 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6a458c91-9b24-42b2-a8a4-a2ab8f4fa0b0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:11.172563 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6a458c91-9b24-42b2-a8a4-a2ab8f4fa0b0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:11.177364 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6a458c91-9b24-42b2-a8a4-a2ab8f4fa0b0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:11.177364 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6a458c91-9b24-42b2-a8a4-a2ab8f4fa0b0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:11.177364 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6a458c91-9b24-42b2-a8a4-a2ab8f4fa0b0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:11.177364 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6a458c91-9b24-42b2-a8a4-a2ab8f4fa0b0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:11.332364 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6a458c91-9b24-42b2-a8a4-a2ab8f4fa0b0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:11.342394 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6a458c91-9b24-42b2-a8a4-a2ab8f4fa0b0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.180787 Aug 21 01:26:11.343519 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 101/194] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:11 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:12.366676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1eb9a865-e2ad-4aa0-affd-b6a0e8aa32e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:12.376155 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1eb9a865-e2ad-4aa0-affd-b6a0e8aa32e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:12.391960 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eb9a865-e2ad-4aa0-affd-b6a0e8aa32e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:12.392478 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eb9a865-e2ad-4aa0-affd-b6a0e8aa32e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:12.392914 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eb9a865-e2ad-4aa0-affd-b6a0e8aa32e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:12.393615 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eb9a865-e2ad-4aa0-affd-b6a0e8aa32e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:12.394136 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eb9a865-e2ad-4aa0-affd-b6a0e8aa32e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:12.394573 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eb9a865-e2ad-4aa0-affd-b6a0e8aa32e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:12.548886 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1eb9a865-e2ad-4aa0-affd-b6a0e8aa32e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:12.558718 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1eb9a865-e2ad-4aa0-affd-b6a0e8aa32e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.197480 Aug 21 01:26:12.559284 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 94/195] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:12 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:13.579925 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-51fd4e08-48f8-473c-9230-a1b7117ad5f2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:13.581817 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-51fd4e08-48f8-473c-9230-a1b7117ad5f2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:13.588465 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51fd4e08-48f8-473c-9230-a1b7117ad5f2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:13.589058 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51fd4e08-48f8-473c-9230-a1b7117ad5f2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:13.589562 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51fd4e08-48f8-473c-9230-a1b7117ad5f2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:13.590619 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51fd4e08-48f8-473c-9230-a1b7117ad5f2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:13.591138 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51fd4e08-48f8-473c-9230-a1b7117ad5f2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:13.592389 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51fd4e08-48f8-473c-9230-a1b7117ad5f2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:13.811565 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-51fd4e08-48f8-473c-9230-a1b7117ad5f2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:13.820793 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-51fd4e08-48f8-473c-9230-a1b7117ad5f2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.244664 Aug 21 01:26:13.821488 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 102/196] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:13 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:14.845894 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-eb372235-d0d3-43f5-b4d7-3f531bff4887 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:14.847160 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-eb372235-d0d3-43f5-b4d7-3f531bff4887 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:14.854232 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb372235-d0d3-43f5-b4d7-3f531bff4887 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:14.854734 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb372235-d0d3-43f5-b4d7-3f531bff4887 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:14.855166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb372235-d0d3-43f5-b4d7-3f531bff4887 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:14.856711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb372235-d0d3-43f5-b4d7-3f531bff4887 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:14.856711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb372235-d0d3-43f5-b4d7-3f531bff4887 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:14.856977 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb372235-d0d3-43f5-b4d7-3f531bff4887 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:15.006695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-eb372235-d0d3-43f5-b4d7-3f531bff4887 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:15.014977 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-eb372235-d0d3-43f5-b4d7-3f531bff4887 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.172819 Aug 21 01:26:15.015555 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 95/197] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:14 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:16.036413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a76abfd9-1ba0-4446-b4b1-9c5df9c2ba0d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:16.037887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a76abfd9-1ba0-4446-b4b1-9c5df9c2ba0d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:16.044852 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a76abfd9-1ba0-4446-b4b1-9c5df9c2ba0d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:16.045257 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a76abfd9-1ba0-4446-b4b1-9c5df9c2ba0d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:16.046044 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a76abfd9-1ba0-4446-b4b1-9c5df9c2ba0d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:16.047218 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a76abfd9-1ba0-4446-b4b1-9c5df9c2ba0d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:16.048363 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a76abfd9-1ba0-4446-b4b1-9c5df9c2ba0d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:16.048808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a76abfd9-1ba0-4446-b4b1-9c5df9c2ba0d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:16.246832 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a76abfd9-1ba0-4446-b4b1-9c5df9c2ba0d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:16.254796 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a76abfd9-1ba0-4446-b4b1-9c5df9c2ba0d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.222861 Aug 21 01:26:16.263086 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 103/198] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:16 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:17.274912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a5bd82d8-18eb-4140-ab28-34a3f7703a97 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:17.275563 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a5bd82d8-18eb-4140-ab28-34a3f7703a97 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:17.281719 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5bd82d8-18eb-4140-ab28-34a3f7703a97 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:17.282202 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5bd82d8-18eb-4140-ab28-34a3f7703a97 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:17.282673 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5bd82d8-18eb-4140-ab28-34a3f7703a97 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:17.283367 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5bd82d8-18eb-4140-ab28-34a3f7703a97 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:17.283824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5bd82d8-18eb-4140-ab28-34a3f7703a97 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:17.284400 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5bd82d8-18eb-4140-ab28-34a3f7703a97 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:17.425949 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a5bd82d8-18eb-4140-ab28-34a3f7703a97 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:17.434266 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a5bd82d8-18eb-4140-ab28-34a3f7703a97 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.162966 Aug 21 01:26:17.434844 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 96/199] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:17 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:18.089523 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-51c2e962-344d-483a-8d0a-430e080e6e49 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:18.090056 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-51c2e962-344d-483a-8d0a-430e080e6e49 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:18.096289 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51c2e962-344d-483a-8d0a-430e080e6e49 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:18.096885 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51c2e962-344d-483a-8d0a-430e080e6e49 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:18.097668 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51c2e962-344d-483a-8d0a-430e080e6e49 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:18.098942 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51c2e962-344d-483a-8d0a-430e080e6e49 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:18.099394 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51c2e962-344d-483a-8d0a-430e080e6e49 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:18.099807 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51c2e962-344d-483a-8d0a-430e080e6e49 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:18.120442 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:26:18.128347 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-51c2e962-344d-483a-8d0a-430e080e6e49 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Going to try to terminate instance {{(pid=121277) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:26:18.455161 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2ee1b799-15f0-4e8f-a7c6-056c37ab3c71 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:18.455769 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2ee1b799-15f0-4e8f-a7c6-056c37ab3c71 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:18.464853 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2ee1b799-15f0-4e8f-a7c6-056c37ab3c71 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:18.465351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2ee1b799-15f0-4e8f-a7c6-056c37ab3c71 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:18.465791 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2ee1b799-15f0-4e8f-a7c6-056c37ab3c71 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:18.466503 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2ee1b799-15f0-4e8f-a7c6-056c37ab3c71 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:18.466955 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2ee1b799-15f0-4e8f-a7c6-056c37ab3c71 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:18.467567 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2ee1b799-15f0-4e8f-a7c6-056c37ab3c71 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:18.554199 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-51c2e962-344d-483a-8d0a-430e080e6e49 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "DELETE /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 204 len: 0 microversion: 2.1 time: 0.465552 Aug 21 01:26:18.554944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 104/200] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:26:18 2022] DELETE /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 0 bytes in 470 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:26:18.568989 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f2dd0068-b12a-4274-b465-a789ba6799c6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:18.569906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f2dd0068-b12a-4274-b465-a789ba6799c6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:18.578677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2dd0068-b12a-4274-b465-a789ba6799c6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:18.579295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2dd0068-b12a-4274-b465-a789ba6799c6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:18.579811 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2dd0068-b12a-4274-b465-a789ba6799c6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:18.580634 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2dd0068-b12a-4274-b465-a789ba6799c6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:18.581173 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2dd0068-b12a-4274-b465-a789ba6799c6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:18.581754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2dd0068-b12a-4274-b465-a789ba6799c6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:18.671417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2ee1b799-15f0-4e8f-a7c6-056c37ab3c71 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:18.683313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2ee1b799-15f0-4e8f-a7c6-056c37ab3c71 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.231888 Aug 21 01:26:18.683928 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 97/201] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:18 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:18.745359 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f2dd0068-b12a-4274-b465-a789ba6799c6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:18.754720 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f2dd0068-b12a-4274-b465-a789ba6799c6 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1554 microversion: 2.1 time: 0.191823 Aug 21 01:26:18.755301 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 105/202] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:18 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1554 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:19.705140 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3869b269-e16d-4b34-ac5a-f81d72ed7730 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:19.705752 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3869b269-e16d-4b34-ac5a-f81d72ed7730 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:19.712355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3869b269-e16d-4b34-ac5a-f81d72ed7730 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:19.712846 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3869b269-e16d-4b34-ac5a-f81d72ed7730 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:19.713283 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3869b269-e16d-4b34-ac5a-f81d72ed7730 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:19.714001 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3869b269-e16d-4b34-ac5a-f81d72ed7730 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:19.714451 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3869b269-e16d-4b34-ac5a-f81d72ed7730 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:19.714874 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3869b269-e16d-4b34-ac5a-f81d72ed7730 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:19.719026 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:26:19.776200 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-90380600-61e8-4c81-8f86-8721a122835e tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:19.776903 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-90380600-61e8-4c81-8f86-8721a122835e tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:19.786183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-90380600-61e8-4c81-8f86-8721a122835e tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:19.787018 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-90380600-61e8-4c81-8f86-8721a122835e tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:19.787692 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-90380600-61e8-4c81-8f86-8721a122835e tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:19.788632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-90380600-61e8-4c81-8f86-8721a122835e tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:19.789159 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-90380600-61e8-4c81-8f86-8721a122835e tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:19.789673 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-90380600-61e8-4c81-8f86-8721a122835e tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:19.907844 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3869b269-e16d-4b34-ac5a-f81d72ed7730 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:19.917955 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3869b269-e16d-4b34-ac5a-f81d72ed7730 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.216733 Aug 21 01:26:19.931211 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 98/203] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:19 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:19.959416 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-90380600-61e8-4c81-8f86-8721a122835e tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:19.968061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-90380600-61e8-4c81-8f86-8721a122835e tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1554 microversion: 2.1 time: 0.195881 Aug 21 01:26:19.968710 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 106/204] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:19 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1554 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:26:20.938930 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b3454239-d4c4-4bc1-8655-bbe1cef1d992 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:20.939446 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-b3454239-d4c4-4bc1-8655-bbe1cef1d992 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:20.946569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b3454239-d4c4-4bc1-8655-bbe1cef1d992 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:20.947077 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b3454239-d4c4-4bc1-8655-bbe1cef1d992 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:20.947507 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b3454239-d4c4-4bc1-8655-bbe1cef1d992 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:20.948404 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b3454239-d4c4-4bc1-8655-bbe1cef1d992 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:20.948855 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b3454239-d4c4-4bc1-8655-bbe1cef1d992 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:20.949276 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b3454239-d4c4-4bc1-8655-bbe1cef1d992 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:20.989196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f199791d-a914-41fb-8176-31103297ed62 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:20.989794 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f199791d-a914-41fb-8176-31103297ed62 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:20.996351 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f199791d-a914-41fb-8176-31103297ed62 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:20.996849 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f199791d-a914-41fb-8176-31103297ed62 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:20.997297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f199791d-a914-41fb-8176-31103297ed62 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:20.997998 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f199791d-a914-41fb-8176-31103297ed62 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:20.998459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f199791d-a914-41fb-8176-31103297ed62 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:20.998891 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f199791d-a914-41fb-8176-31103297ed62 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:21.125372 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b3454239-d4c4-4bc1-8655-bbe1cef1d992 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:21.134631 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b3454239-d4c4-4bc1-8655-bbe1cef1d992 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.199552 Aug 21 01:26:21.135217 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 99/205] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:20 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:21.196117 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f199791d-a914-41fb-8176-31103297ed62 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:21.204344 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f199791d-a914-41fb-8176-31103297ed62 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1554 microversion: 2.1 time: 0.219427 Aug 21 01:26:21.204970 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 107/206] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:20 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1554 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:26:21.870489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-201a5809-2997-49e4-8118-6d45fb06a1ec req-c921f6f0-8319-47fb-86ba-e329fe8f810d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "031af209-ae76-47b8-8481-88ec4053affa", "name": "network-vif-deleted", "tag": "667f35ba-e131-4619-82ff-965a65cd85fd"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:26:21.886089 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-201a5809-2997-49e4-8118-6d45fb06a1ec req-c921f6f0-8319-47fb-86ba-e329fe8f810d service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:21.886798 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-201a5809-2997-49e4-8118-6d45fb06a1ec req-c921f6f0-8319-47fb-86ba-e329fe8f810d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:21.887246 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-201a5809-2997-49e4-8118-6d45fb06a1ec req-c921f6f0-8319-47fb-86ba-e329fe8f810d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:21.900286 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-201a5809-2997-49e4-8118-6d45fb06a1ec req-c921f6f0-8319-47fb-86ba-e329fe8f810d service nova] Creating event network-vif-deleted:667f35ba-e131-4619-82ff-965a65cd85fd for instance 031af209-ae76-47b8-8481-88ec4053affa on hv-cinder-maste Aug 21 01:26:21.905561 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-201a5809-2997-49e4-8118-6d45fb06a1ec req-c921f6f0-8319-47fb-86ba-e329fe8f810d service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038818 Aug 21 01:26:21.908535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 100/207] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:26:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:26:22.156456 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3d330c04-4636-41a7-a46e-6b636715285e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:22.157061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3d330c04-4636-41a7-a46e-6b636715285e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:22.164196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d330c04-4636-41a7-a46e-6b636715285e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:22.164836 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d330c04-4636-41a7-a46e-6b636715285e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:22.165282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d330c04-4636-41a7-a46e-6b636715285e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:22.166016 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d330c04-4636-41a7-a46e-6b636715285e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:22.166467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d330c04-4636-41a7-a46e-6b636715285e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:22.166886 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d330c04-4636-41a7-a46e-6b636715285e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:22.227794 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-01802dcf-bf02-432c-9cf8-fcc75167fae1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:22.228636 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-01802dcf-bf02-432c-9cf8-fcc75167fae1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:22.235070 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-01802dcf-bf02-432c-9cf8-fcc75167fae1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:22.235561 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-01802dcf-bf02-432c-9cf8-fcc75167fae1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:22.236000 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-01802dcf-bf02-432c-9cf8-fcc75167fae1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:22.237771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-01802dcf-bf02-432c-9cf8-fcc75167fae1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:22.237771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-01802dcf-bf02-432c-9cf8-fcc75167fae1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:22.237771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-01802dcf-bf02-432c-9cf8-fcc75167fae1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:22.314986 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-01802dcf-bf02-432c-9cf8-fcc75167fae1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:22.323212 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-01802dcf-bf02-432c-9cf8-fcc75167fae1 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1346 microversion: 2.1 time: 0.100758 Aug 21 01:26:22.323783 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 101/208] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:22 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1346 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:26:22.329192 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3d330c04-4636-41a7-a46e-6b636715285e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:22.337897 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3d330c04-4636-41a7-a46e-6b636715285e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.185257 Aug 21 01:26:22.338488 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 108/209] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:22 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:23.345819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:23.346956 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:23.361310 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-68709046-0bcc-450d-a827-48d930ded959 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:23.362124 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-68709046-0bcc-450d-a827-48d930ded959 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:23.364153 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:23.364675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:23.365111 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:23.365803 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:23.366396 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:23.366958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:23.369360 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68709046-0bcc-450d-a827-48d930ded959 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:23.369850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68709046-0bcc-450d-a827-48d930ded959 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:23.370325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68709046-0bcc-450d-a827-48d930ded959 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:23.371099 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68709046-0bcc-450d-a827-48d930ded959 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:23.371557 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68709046-0bcc-450d-a827-48d930ded959 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:23.372186 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68709046-0bcc-450d-a827-48d930ded959 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:23.400374 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:23.400860 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:23.401295 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:23.401749 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lazy-loading 'fault' on Instance uuid 031af209-ae76-47b8-8481-88ec4053affa {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:26:23.469573 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc7137d0ba634b6ab4b83de1ab47c476', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1914f8232864af6ad97aebcb1636125', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:23.478590 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f1c962ba-9606-42fd-8470-f8e05fde3bb5 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 200 len: 1351 microversion: 2.1 time: 0.137624 Aug 21 01:26:23.479174 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 102/210] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:23 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 1351 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:26:23.569097 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-68709046-0bcc-450d-a827-48d930ded959 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:23.569097 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-68709046-0bcc-450d-a827-48d930ded959 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.212378 Aug 21 01:26:23.570823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 109/211] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:23 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:24.499821 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-21e85f9e-128f-451f-ad07-ddb716cd9e13 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:24.500476 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-21e85f9e-128f-451f-ad07-ddb716cd9e13 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:24.507471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-21e85f9e-128f-451f-ad07-ddb716cd9e13 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:24.507961 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-21e85f9e-128f-451f-ad07-ddb716cd9e13 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:24.508440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-21e85f9e-128f-451f-ad07-ddb716cd9e13 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:24.509144 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-21e85f9e-128f-451f-ad07-ddb716cd9e13 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:24.509599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-21e85f9e-128f-451f-ad07-ddb716cd9e13 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:24.510021 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-21e85f9e-128f-451f-ad07-ddb716cd9e13 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:24.526518 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-21e85f9e-128f-451f-ad07-ddb716cd9e13 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] HTTP exception thrown: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-21e85f9e-128f-451f-ad07-ddb716cd9e13 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Request method failure captured: Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa HTTP/1.1 Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYl3_j_DGAUdG_l8WiN5wnG0hjXc-Yeh7xYPbW2kmnwVD_4n6r7zxaM_1tq7souJoL9kP_YxInIRMr0hDU7l8lY_RVVFah93wBDUUb4nwxB-y908IWrZjhJKwW8wGwjnU8EusJftQaa4DrwLnrGMNR6aNUrxvWDFFw14NPE4j1DUdKo Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: a1914f8232864af6ad97aebcb1636125 Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-DeleteServersTestJSON-919690758 Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:26:24.531183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:26:24.532938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/a1914f8232864af6ad97aebcb1636125"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_a1914f8232864af6ad97aebcb1636125"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/a1914f8232864af6ad97aebcb1636125"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/a1914f8232864af6ad97aebcb1636125"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:26:24.532938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-DeleteServersTestJSON-919690758 Aug 21 01:26:24.532938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: a1914f8232864af6ad97aebcb1636125 Aug 21 01:26:24.532938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-DeleteServersTestJSON-919690758 Aug 21 01:26:24.532938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-DeleteServersTestJSON-919690758-project Aug 21 01:26:24.532938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:26:24.532938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:26:24.532938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: bc7137d0ba634b6ab4b83de1ab47c476 Aug 21 01:26:24.532938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-DeleteServersTestJSON-919690758-project Aug 21 01:26:24.532938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:26:24.532938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:24.532938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '031af209-ae76-47b8-8481-88ec4053affa'} Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:26:24.533725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:24.535695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:24.537218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-21e85f9e-128f-451f-ad07-ddb716cd9e13 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Returning 404 to user: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:26:24.537218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-21e85f9e-128f-451f-ad07-ddb716cd9e13 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 404 len: 111 microversion: 2.1 time: 0.036767 Aug 21 01:26:24.537218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 103/212] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:24 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:26:24.595204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ad06ccc6-2898-415a-80f3-317e79d97fbf tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:24.596217 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-ad06ccc6-2898-415a-80f3-317e79d97fbf tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:24.605330 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ad06ccc6-2898-415a-80f3-317e79d97fbf tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:24.606103 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ad06ccc6-2898-415a-80f3-317e79d97fbf tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:24.606759 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ad06ccc6-2898-415a-80f3-317e79d97fbf tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:24.607554 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ad06ccc6-2898-415a-80f3-317e79d97fbf tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:24.608428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ad06ccc6-2898-415a-80f3-317e79d97fbf tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:24.608945 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ad06ccc6-2898-415a-80f3-317e79d97fbf tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:24.652379 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4ce02bb1-0b69-456a-a935-29ccaac7aa93 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:24.654770 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4ce02bb1-0b69-456a-a935-29ccaac7aa93 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:24.663369 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ce02bb1-0b69-456a-a935-29ccaac7aa93 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:24.663883 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ce02bb1-0b69-456a-a935-29ccaac7aa93 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:24.664349 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ce02bb1-0b69-456a-a935-29ccaac7aa93 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:24.665304 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ce02bb1-0b69-456a-a935-29ccaac7aa93 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:24.665748 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ce02bb1-0b69-456a-a935-29ccaac7aa93 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:24.666335 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ce02bb1-0b69-456a-a935-29ccaac7aa93 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:24.688636 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-4ce02bb1-0b69-456a-a935-29ccaac7aa93 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] HTTP exception thrown: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4ce02bb1-0b69-456a-a935-29ccaac7aa93 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Request method failure captured: Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa/os-volume_attachments HTTP/1.1 Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYl3_j_DGAUdG_l8WiN5wnG0hjXc-Yeh7xYPbW2kmnwVD_4n6r7zxaM_1tq7souJoL9kP_YxInIRMr0hDU7l8lY_RVVFah93wBDUUb4nwxB-y908IWrZjhJKwW8wGwjnU8EusJftQaa4DrwLnrGMNR6aNUrxvWDFFw14NPE4j1DUdKo Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: a1914f8232864af6ad97aebcb1636125 Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-DeleteServersTestJSON-919690758 Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:26:24.694677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:26:24.696635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/a1914f8232864af6ad97aebcb1636125"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_a1914f8232864af6ad97aebcb1636125"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/a1914f8232864af6ad97aebcb1636125"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/a1914f8232864af6ad97aebcb1636125"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:26:24.696635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-DeleteServersTestJSON-919690758 Aug 21 01:26:24.696635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: a1914f8232864af6ad97aebcb1636125 Aug 21 01:26:24.696635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-DeleteServersTestJSON-919690758 Aug 21 01:26:24.696635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-DeleteServersTestJSON-919690758-project Aug 21 01:26:24.696635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:26:24.696635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:26:24.696635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: bc7137d0ba634b6ab4b83de1ab47c476 Aug 21 01:26:24.696635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-DeleteServersTestJSON-919690758-project Aug 21 01:26:24.696635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:26:24.696635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:24.696635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'server_id': '031af209-ae76-47b8-8481-88ec4053affa'} Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:26:24.697322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:24.700442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:24.703793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4ce02bb1-0b69-456a-a935-29ccaac7aa93 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Returning 404 to user: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:26:24.703793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4ce02bb1-0b69-456a-a935-29ccaac7aa93 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.043491 Aug 21 01:26:24.703793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 104/213] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:26:24 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa/os-volume_attachments => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 21 01:26:24.802717 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ad06ccc6-2898-415a-80f3-317e79d97fbf tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:24.813881 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ad06ccc6-2898-415a-80f3-317e79d97fbf tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.222938 Aug 21 01:26:24.814825 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 110/214] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:24 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:25.840726 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-cd356f85-d7ee-417c-9949-88a6f31ce4f7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:25.841660 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-cd356f85-d7ee-417c-9949-88a6f31ce4f7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:25.851818 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cd356f85-d7ee-417c-9949-88a6f31ce4f7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:25.852720 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cd356f85-d7ee-417c-9949-88a6f31ce4f7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:25.853479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cd356f85-d7ee-417c-9949-88a6f31ce4f7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:25.854647 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cd356f85-d7ee-417c-9949-88a6f31ce4f7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:25.855429 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cd356f85-d7ee-417c-9949-88a6f31ce4f7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:25.856206 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cd356f85-d7ee-417c-9949-88a6f31ce4f7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:26.007870 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-cd356f85-d7ee-417c-9949-88a6f31ce4f7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:26.015721 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-cd356f85-d7ee-417c-9949-88a6f31ce4f7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.182203 Aug 21 01:26:26.016358 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 105/215] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:25 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:26.147951 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-95fc42ff-7d27-4cea-932d-ed32bfe69b57 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:26.148534 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-95fc42ff-7d27-4cea-932d-ed32bfe69b57 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:26.154779 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-95fc42ff-7d27-4cea-932d-ed32bfe69b57 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:26.155265 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-95fc42ff-7d27-4cea-932d-ed32bfe69b57 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:26.155698 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-95fc42ff-7d27-4cea-932d-ed32bfe69b57 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:26.156489 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-95fc42ff-7d27-4cea-932d-ed32bfe69b57 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:26.156883 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-95fc42ff-7d27-4cea-932d-ed32bfe69b57 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:26.157308 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-95fc42ff-7d27-4cea-932d-ed32bfe69b57 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:26.172858 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-95fc42ff-7d27-4cea-932d-ed32bfe69b57 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] HTTP exception thrown: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-95fc42ff-7d27-4cea-932d-ed32bfe69b57 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Request method failure captured: Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: DELETE /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa HTTP/1.1 Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYl3_j_DGAUdG_l8WiN5wnG0hjXc-Yeh7xYPbW2kmnwVD_4n6r7zxaM_1tq7souJoL9kP_YxInIRMr0hDU7l8lY_RVVFah93wBDUUb4nwxB-y908IWrZjhJKwW8wGwjnU8EusJftQaa4DrwLnrGMNR6aNUrxvWDFFw14NPE4j1DUdKo Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: a1914f8232864af6ad97aebcb1636125 Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-DeleteServersTestJSON-919690758 Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:26:26.177537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:26:26.179002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/a1914f8232864af6ad97aebcb1636125"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_a1914f8232864af6ad97aebcb1636125"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/a1914f8232864af6ad97aebcb1636125"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/a1914f8232864af6ad97aebcb1636125"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:26:26.179002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-DeleteServersTestJSON-919690758 Aug 21 01:26:26.179002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: a1914f8232864af6ad97aebcb1636125 Aug 21 01:26:26.179002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-DeleteServersTestJSON-919690758 Aug 21 01:26:26.179002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-DeleteServersTestJSON-919690758-project Aug 21 01:26:26.179002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:26:26.179002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:26:26.179002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: bc7137d0ba634b6ab4b83de1ab47c476 Aug 21 01:26:26.179002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-DeleteServersTestJSON-919690758-project Aug 21 01:26:26.179002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:26:26.179002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:26.179002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '031af209-ae76-47b8-8481-88ec4053affa'} Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:26:26.179754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:26.181100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:26:26.182386 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-95fc42ff-7d27-4cea-932d-ed32bfe69b57 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Returning 404 to user: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:26:26.182386 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-95fc42ff-7d27-4cea-932d-ed32bfe69b57 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "DELETE /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 404 len: 111 microversion: 2.1 time: 0.038107 Aug 21 01:26:26.182905 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 111/216] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:26:26 2022] DELETE /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:26:26.189905 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bf3c525a-7c29-490f-8c6f-ef3050c8cb05 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:26.190646 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bf3c525a-7c29-490f-8c6f-ef3050c8cb05 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] [instance: 031af209-ae76-47b8-8481-88ec4053affa] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:26.196862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bf3c525a-7c29-490f-8c6f-ef3050c8cb05 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:26.197354 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bf3c525a-7c29-490f-8c6f-ef3050c8cb05 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:26.197787 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bf3c525a-7c29-490f-8c6f-ef3050c8cb05 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:26.198481 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bf3c525a-7c29-490f-8c6f-ef3050c8cb05 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:26.198973 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bf3c525a-7c29-490f-8c6f-ef3050c8cb05 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:26.199403 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bf3c525a-7c29-490f-8c6f-ef3050c8cb05 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:26.214933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-bf3c525a-7c29-490f-8c6f-ef3050c8cb05 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] HTTP exception thrown: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bf3c525a-7c29-490f-8c6f-ef3050c8cb05 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Request method failure captured: Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa HTTP/1.1 Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYl3_j_DGAUdG_l8WiN5wnG0hjXc-Yeh7xYPbW2kmnwVD_4n6r7zxaM_1tq7souJoL9kP_YxInIRMr0hDU7l8lY_RVVFah93wBDUUb4nwxB-y908IWrZjhJKwW8wGwjnU8EusJftQaa4DrwLnrGMNR6aNUrxvWDFFw14NPE4j1DUdKo Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: a1914f8232864af6ad97aebcb1636125 Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-DeleteServersTestJSON-919690758 Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:26:26.216553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:26:26.218308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/a1914f8232864af6ad97aebcb1636125"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_a1914f8232864af6ad97aebcb1636125"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/a1914f8232864af6ad97aebcb1636125"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/a1914f8232864af6ad97aebcb1636125"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:26:26.218308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-DeleteServersTestJSON-919690758 Aug 21 01:26:26.218308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: a1914f8232864af6ad97aebcb1636125 Aug 21 01:26:26.218308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-DeleteServersTestJSON-919690758 Aug 21 01:26:26.218308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-DeleteServersTestJSON-919690758-project Aug 21 01:26:26.218308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:26:26.218308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:26:26.218308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: bc7137d0ba634b6ab4b83de1ab47c476 Aug 21 01:26:26.218308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-DeleteServersTestJSON-919690758-project Aug 21 01:26:26.218308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:26:26.218308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:26.218308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '031af209-ae76-47b8-8481-88ec4053affa'} Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:26:26.219110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. Aug 21 01:26:26.220411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:26:26.221650 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bf3c525a-7c29-490f-8c6f-ef3050c8cb05 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] Returning 404 to user: Instance 031af209-ae76-47b8-8481-88ec4053affa could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:26:26.221650 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bf3c525a-7c29-490f-8c6f-ef3050c8cb05 tempest-DeleteServersTestJSON-919690758 tempest-DeleteServersTestJSON-919690758-project] 10.222.0.47 "GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa" status: 404 len: 111 microversion: 2.1 time: 0.033697 Aug 21 01:26:26.221650 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 106/217] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:26 2022] GET /compute/v2.1/servers/031af209-ae76-47b8-8481-88ec4053affa => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:26:27.039051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-12ba2a40-db73-4a7a-b5bf-5b5833cc319c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:27.039869 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-12ba2a40-db73-4a7a-b5bf-5b5833cc319c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:27.046120 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-12ba2a40-db73-4a7a-b5bf-5b5833cc319c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:27.046608 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-12ba2a40-db73-4a7a-b5bf-5b5833cc319c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:27.047349 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-12ba2a40-db73-4a7a-b5bf-5b5833cc319c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:27.047864 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-12ba2a40-db73-4a7a-b5bf-5b5833cc319c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:27.048366 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-12ba2a40-db73-4a7a-b5bf-5b5833cc319c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:27.049001 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-12ba2a40-db73-4a7a-b5bf-5b5833cc319c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:27.198865 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-12ba2a40-db73-4a7a-b5bf-5b5833cc319c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:27.207343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-12ba2a40-db73-4a7a-b5bf-5b5833cc319c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.172419 Aug 21 01:26:27.207921 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 112/218] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:27 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:28.228742 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-5756ffa9-c028-49bc-bd19-0e89b9dcb820 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:28.229363 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-5756ffa9-c028-49bc-bd19-0e89b9dcb820 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:28.236281 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5756ffa9-c028-49bc-bd19-0e89b9dcb820 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:28.237072 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5756ffa9-c028-49bc-bd19-0e89b9dcb820 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:28.238237 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5756ffa9-c028-49bc-bd19-0e89b9dcb820 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:28.239143 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5756ffa9-c028-49bc-bd19-0e89b9dcb820 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:28.239792 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5756ffa9-c028-49bc-bd19-0e89b9dcb820 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:28.240438 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5756ffa9-c028-49bc-bd19-0e89b9dcb820 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:28.441576 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5756ffa9-c028-49bc-bd19-0e89b9dcb820 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:28.449599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5756ffa9-c028-49bc-bd19-0e89b9dcb820 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.224795 Aug 21 01:26:28.450025 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 107/219] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:28 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:29.471067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8f6b6a9d-6b3f-4ce7-bc31-8cd166adb343 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:29.473038 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8f6b6a9d-6b3f-4ce7-bc31-8cd166adb343 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:29.482137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8f6b6a9d-6b3f-4ce7-bc31-8cd166adb343 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:29.482442 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8f6b6a9d-6b3f-4ce7-bc31-8cd166adb343 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:29.483196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8f6b6a9d-6b3f-4ce7-bc31-8cd166adb343 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:29.484068 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8f6b6a9d-6b3f-4ce7-bc31-8cd166adb343 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:29.484712 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8f6b6a9d-6b3f-4ce7-bc31-8cd166adb343 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:29.485469 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8f6b6a9d-6b3f-4ce7-bc31-8cd166adb343 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:29.678468 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8f6b6a9d-6b3f-4ce7-bc31-8cd166adb343 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:29.686431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8f6b6a9d-6b3f-4ce7-bc31-8cd166adb343 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.219386 Aug 21 01:26:29.686998 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 113/220] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:29 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:30.707398 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d3898fca-9009-4bf0-b605-e537a8903853 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:30.708000 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d3898fca-9009-4bf0-b605-e537a8903853 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:30.710458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=58 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:26:30.716318 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d3898fca-9009-4bf0-b605-e537a8903853 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:30.716797 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d3898fca-9009-4bf0-b605-e537a8903853 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:30.717237 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d3898fca-9009-4bf0-b605-e537a8903853 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:30.717921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d3898fca-9009-4bf0-b605-e537a8903853 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:30.718505 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d3898fca-9009-4bf0-b605-e537a8903853 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:30.718925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d3898fca-9009-4bf0-b605-e537a8903853 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:31.029442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d3898fca-9009-4bf0-b605-e537a8903853 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:31.037835 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d3898fca-9009-4bf0-b605-e537a8903853 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.334743 Aug 21 01:26:31.038433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 108/221] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:30 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:26:32.058899 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-93778df5-d4d0-4456-87ea-503684804b79 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:32.060141 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-93778df5-d4d0-4456-87ea-503684804b79 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:32.066886 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-93778df5-d4d0-4456-87ea-503684804b79 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:32.067515 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-93778df5-d4d0-4456-87ea-503684804b79 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:32.068026 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-93778df5-d4d0-4456-87ea-503684804b79 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:32.069090 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-93778df5-d4d0-4456-87ea-503684804b79 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:32.069623 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-93778df5-d4d0-4456-87ea-503684804b79 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:32.070118 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-93778df5-d4d0-4456-87ea-503684804b79 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:32.223619 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-93778df5-d4d0-4456-87ea-503684804b79 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:32.231885 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-93778df5-d4d0-4456-87ea-503684804b79 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.177062 Aug 21 01:26:32.232512 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 114/222] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:32 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:33.252090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2171d400-9e71-4ea2-afef-632816b7495e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:33.252737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2171d400-9e71-4ea2-afef-632816b7495e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:33.259217 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2171d400-9e71-4ea2-afef-632816b7495e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:33.259709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2171d400-9e71-4ea2-afef-632816b7495e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:33.260173 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2171d400-9e71-4ea2-afef-632816b7495e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:33.260876 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2171d400-9e71-4ea2-afef-632816b7495e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:33.261328 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2171d400-9e71-4ea2-afef-632816b7495e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:33.261754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2171d400-9e71-4ea2-afef-632816b7495e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:33.400817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2171d400-9e71-4ea2-afef-632816b7495e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:33.409280 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2171d400-9e71-4ea2-afef-632816b7495e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.160611 Aug 21 01:26:33.410103 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 109/223] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:33 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:34.431333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-28d11eac-153b-4ad7-b9cb-2ce185883873 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:34.432604 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-28d11eac-153b-4ad7-b9cb-2ce185883873 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:34.439178 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-28d11eac-153b-4ad7-b9cb-2ce185883873 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:34.439685 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-28d11eac-153b-4ad7-b9cb-2ce185883873 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:34.440134 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-28d11eac-153b-4ad7-b9cb-2ce185883873 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:34.440862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-28d11eac-153b-4ad7-b9cb-2ce185883873 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:34.441320 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-28d11eac-153b-4ad7-b9cb-2ce185883873 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:34.441813 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-28d11eac-153b-4ad7-b9cb-2ce185883873 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:34.663756 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-28d11eac-153b-4ad7-b9cb-2ce185883873 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:34.675262 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-28d11eac-153b-4ad7-b9cb-2ce185883873 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.247123 Aug 21 01:26:34.675954 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 115/224] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:34 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:35.700919 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f03d043c-82a1-4a85-be19-cfc36f48e6c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:35.703040 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f03d043c-82a1-4a85-be19-cfc36f48e6c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:35.712902 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f03d043c-82a1-4a85-be19-cfc36f48e6c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:35.714472 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f03d043c-82a1-4a85-be19-cfc36f48e6c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:35.716338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f03d043c-82a1-4a85-be19-cfc36f48e6c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:35.717911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f03d043c-82a1-4a85-be19-cfc36f48e6c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:35.718573 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f03d043c-82a1-4a85-be19-cfc36f48e6c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:35.722564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f03d043c-82a1-4a85-be19-cfc36f48e6c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:35.891854 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f03d043c-82a1-4a85-be19-cfc36f48e6c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:35.902933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f03d043c-82a1-4a85-be19-cfc36f48e6c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.208558 Aug 21 01:26:35.904575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 110/225] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:35 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:36.926741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-0b3ede1a-6d4c-4dd5-a8e1-495f67821f70 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:36.927992 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-0b3ede1a-6d4c-4dd5-a8e1-495f67821f70 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:36.934735 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0b3ede1a-6d4c-4dd5-a8e1-495f67821f70 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:36.935817 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0b3ede1a-6d4c-4dd5-a8e1-495f67821f70 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:36.936295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0b3ede1a-6d4c-4dd5-a8e1-495f67821f70 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:36.936995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0b3ede1a-6d4c-4dd5-a8e1-495f67821f70 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:36.937444 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0b3ede1a-6d4c-4dd5-a8e1-495f67821f70 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:36.938028 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0b3ede1a-6d4c-4dd5-a8e1-495f67821f70 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:37.112422 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-0b3ede1a-6d4c-4dd5-a8e1-495f67821f70 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:37.123364 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-0b3ede1a-6d4c-4dd5-a8e1-495f67821f70 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.200624 Aug 21 01:26:37.124135 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 116/226] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:36 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:38.144714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-79289f46-c9a9-47f8-82ba-399d3296024e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:38.145328 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-79289f46-c9a9-47f8-82ba-399d3296024e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:38.151777 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-79289f46-c9a9-47f8-82ba-399d3296024e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:38.152327 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-79289f46-c9a9-47f8-82ba-399d3296024e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:38.152770 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-79289f46-c9a9-47f8-82ba-399d3296024e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:38.153454 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-79289f46-c9a9-47f8-82ba-399d3296024e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:38.154083 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-79289f46-c9a9-47f8-82ba-399d3296024e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:38.154509 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-79289f46-c9a9-47f8-82ba-399d3296024e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:38.331194 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-79289f46-c9a9-47f8-82ba-399d3296024e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:38.341082 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-79289f46-c9a9-47f8-82ba-399d3296024e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.200052 Aug 21 01:26:38.357104 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 111/227] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:38 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:39.362861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d1837b1c-4677-4da3-a79e-3703d51fb9ea tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:39.363601 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d1837b1c-4677-4da3-a79e-3703d51fb9ea tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:39.365711 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=48,nova_api:UPDATE=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:26:39.370607 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1837b1c-4677-4da3-a79e-3703d51fb9ea tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:39.371095 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1837b1c-4677-4da3-a79e-3703d51fb9ea tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:39.371608 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1837b1c-4677-4da3-a79e-3703d51fb9ea tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:39.372414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1837b1c-4677-4da3-a79e-3703d51fb9ea tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:39.372879 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1837b1c-4677-4da3-a79e-3703d51fb9ea tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:39.373303 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1837b1c-4677-4da3-a79e-3703d51fb9ea tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:39.613159 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d1837b1c-4677-4da3-a79e-3703d51fb9ea tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:39.628276 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d1837b1c-4677-4da3-a79e-3703d51fb9ea tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.269223 Aug 21 01:26:39.628915 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 117/228] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:39 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:40.662578 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-365c4323-b823-480a-90b3-73efc5514c3a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:40.663191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-365c4323-b823-480a-90b3-73efc5514c3a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:40.670263 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-365c4323-b823-480a-90b3-73efc5514c3a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:40.670752 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-365c4323-b823-480a-90b3-73efc5514c3a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:40.671197 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-365c4323-b823-480a-90b3-73efc5514c3a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:40.672028 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-365c4323-b823-480a-90b3-73efc5514c3a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:40.672510 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-365c4323-b823-480a-90b3-73efc5514c3a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:40.672944 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-365c4323-b823-480a-90b3-73efc5514c3a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:40.846261 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-365c4323-b823-480a-90b3-73efc5514c3a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:40.854485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-365c4323-b823-480a-90b3-73efc5514c3a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.196591 Aug 21 01:26:40.855254 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 112/229] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:40 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:41.877069 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8fa11255-f266-4d0d-8196-c6331de43068 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:41.877690 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8fa11255-f266-4d0d-8196-c6331de43068 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:41.884577 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8fa11255-f266-4d0d-8196-c6331de43068 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:41.885062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8fa11255-f266-4d0d-8196-c6331de43068 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:41.885504 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8fa11255-f266-4d0d-8196-c6331de43068 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:41.886198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8fa11255-f266-4d0d-8196-c6331de43068 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:41.886654 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8fa11255-f266-4d0d-8196-c6331de43068 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:41.887074 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8fa11255-f266-4d0d-8196-c6331de43068 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:42.044548 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8fa11255-f266-4d0d-8196-c6331de43068 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:42.054042 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8fa11255-f266-4d0d-8196-c6331de43068 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.181080 Aug 21 01:26:42.054598 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 118/230] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:41 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:43.076462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b385db51-0919-4526-91db-1302675a9424 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:43.077822 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-b385db51-0919-4526-91db-1302675a9424 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:43.089165 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b385db51-0919-4526-91db-1302675a9424 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:43.089875 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b385db51-0919-4526-91db-1302675a9424 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:43.090542 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b385db51-0919-4526-91db-1302675a9424 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:43.092819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b385db51-0919-4526-91db-1302675a9424 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:43.093521 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b385db51-0919-4526-91db-1302675a9424 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:43.094049 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b385db51-0919-4526-91db-1302675a9424 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:43.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b385db51-0919-4526-91db-1302675a9424 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:43.276532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b385db51-0919-4526-91db-1302675a9424 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1521 microversion: 2.1 time: 0.205082 Aug 21 01:26:43.277131 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 113/231] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:43 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1521 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:44.298241 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d83b3d7e-8104-41a4-8abe-53fa94df3808 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:44.298846 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d83b3d7e-8104-41a4-8abe-53fa94df3808 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:44.305119 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d83b3d7e-8104-41a4-8abe-53fa94df3808 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:44.305598 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d83b3d7e-8104-41a4-8abe-53fa94df3808 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:44.306136 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d83b3d7e-8104-41a4-8abe-53fa94df3808 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:44.306812 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d83b3d7e-8104-41a4-8abe-53fa94df3808 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:44.307259 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d83b3d7e-8104-41a4-8abe-53fa94df3808 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:44.307680 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d83b3d7e-8104-41a4-8abe-53fa94df3808 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:44.454203 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d83b3d7e-8104-41a4-8abe-53fa94df3808 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:44.464270 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d83b3d7e-8104-41a4-8abe-53fa94df3808 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1501 microversion: 2.1 time: 0.170182 Aug 21 01:26:44.465270 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 119/232] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:44 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1501 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:44.484828 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8e9fb9fa-ae71-4129-8526-c6a33922fbd6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:44.485563 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8e9fb9fa-ae71-4129-8526-c6a33922fbd6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:44.492462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e9fb9fa-ae71-4129-8526-c6a33922fbd6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:44.496754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e9fb9fa-ae71-4129-8526-c6a33922fbd6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:44.496754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e9fb9fa-ae71-4129-8526-c6a33922fbd6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:44.496754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e9fb9fa-ae71-4129-8526-c6a33922fbd6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:44.496754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e9fb9fa-ae71-4129-8526-c6a33922fbd6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:44.497163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e9fb9fa-ae71-4129-8526-c6a33922fbd6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:44.632162 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8e9fb9fa-ae71-4129-8526-c6a33922fbd6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:44.673890 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8e9fb9fa-ae71-4129-8526-c6a33922fbd6 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1516 microversion: 2.1 time: 0.193234 Aug 21 01:26:44.674661 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 114/233] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:44 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1516 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:46.645977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9ed02521-400d-4b77-90e9-7cd2b7a0f37b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:26:46.648612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9ed02521-400d-4b77-90e9-7cd2b7a0f37b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:46.655101 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ed02521-400d-4b77-90e9-7cd2b7a0f37b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:46.655728 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ed02521-400d-4b77-90e9-7cd2b7a0f37b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:46.656340 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ed02521-400d-4b77-90e9-7cd2b7a0f37b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:46.657125 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ed02521-400d-4b77-90e9-7cd2b7a0f37b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:46.657588 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ed02521-400d-4b77-90e9-7cd2b7a0f37b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:46.658049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ed02521-400d-4b77-90e9-7cd2b7a0f37b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:46.662325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:26:46.683942 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-9ed02521-400d-4b77-90e9-7cd2b7a0f37b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lazy-loading 'flavor' on Instance uuid 90e88110-1175-4ccf-bf6b-8efb43ab945e {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:26:47.112789 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9ed02521-400d-4b77-90e9-7cd2b7a0f37b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "POST /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e/action" status: 200 len: 32 microversion: 2.1 time: 0.473242 Aug 21 01:26:47.113429 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 120/234] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:26:46 2022] POST /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e/action => generated 32 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 21 01:26:47.122921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bbed6035-6c86-496b-92df-0afbb76d3c2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:47.123765 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bbed6035-6c86-496b-92df-0afbb76d3c2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:47.130614 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bbed6035-6c86-496b-92df-0afbb76d3c2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:47.131108 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bbed6035-6c86-496b-92df-0afbb76d3c2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:47.131550 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bbed6035-6c86-496b-92df-0afbb76d3c2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:47.132284 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bbed6035-6c86-496b-92df-0afbb76d3c2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:47.132889 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bbed6035-6c86-496b-92df-0afbb76d3c2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:47.133246 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bbed6035-6c86-496b-92df-0afbb76d3c2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:47.281870 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-bbed6035-6c86-496b-92df-0afbb76d3c2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:47.290610 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bbed6035-6c86-496b-92df-0afbb76d3c2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.171484 Aug 21 01:26:47.291187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 115/235] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:47 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:48.311199 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d815eb38-d99d-45c8-a789-68443d6360a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:48.312037 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d815eb38-d99d-45c8-a789-68443d6360a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:48.321886 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d815eb38-d99d-45c8-a789-68443d6360a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:48.321886 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d815eb38-d99d-45c8-a789-68443d6360a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:48.321886 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d815eb38-d99d-45c8-a789-68443d6360a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:48.321886 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d815eb38-d99d-45c8-a789-68443d6360a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:48.322230 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d815eb38-d99d-45c8-a789-68443d6360a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:48.322230 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d815eb38-d99d-45c8-a789-68443d6360a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:48.469108 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d815eb38-d99d-45c8-a789-68443d6360a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:48.478578 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d815eb38-d99d-45c8-a789-68443d6360a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.171437 Aug 21 01:26:48.479369 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 121/236] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:48 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:49.498912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2cd791fd-7d41-411b-b329-28f3373c770b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:49.499730 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2cd791fd-7d41-411b-b329-28f3373c770b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:49.506798 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2cd791fd-7d41-411b-b329-28f3373c770b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:49.507330 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2cd791fd-7d41-411b-b329-28f3373c770b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:49.507794 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2cd791fd-7d41-411b-b329-28f3373c770b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:49.508507 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2cd791fd-7d41-411b-b329-28f3373c770b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:49.508964 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2cd791fd-7d41-411b-b329-28f3373c770b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:49.509385 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2cd791fd-7d41-411b-b329-28f3373c770b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:49.722351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2cd791fd-7d41-411b-b329-28f3373c770b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:49.730941 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2cd791fd-7d41-411b-b329-28f3373c770b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.235919 Aug 21 01:26:49.731517 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 116/237] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:49 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:50.752251 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-bf3cea4e-7772-4467-bc10-cdc7faba3752 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:50.752881 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-bf3cea4e-7772-4467-bc10-cdc7faba3752 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:50.759915 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bf3cea4e-7772-4467-bc10-cdc7faba3752 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:50.760425 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bf3cea4e-7772-4467-bc10-cdc7faba3752 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:50.760861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bf3cea4e-7772-4467-bc10-cdc7faba3752 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:50.761811 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bf3cea4e-7772-4467-bc10-cdc7faba3752 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:50.762224 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bf3cea4e-7772-4467-bc10-cdc7faba3752 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:50.762688 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bf3cea4e-7772-4467-bc10-cdc7faba3752 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:50.903338 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-bf3cea4e-7772-4467-bc10-cdc7faba3752 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:50.911616 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-bf3cea4e-7772-4467-bc10-cdc7faba3752 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.163076 Aug 21 01:26:50.912202 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 122/238] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:50 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:51.935085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fe912d8f-42c7-410f-8c5d-d436a41b9f4c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:51.935889 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fe912d8f-42c7-410f-8c5d-d436a41b9f4c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:51.942159 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe912d8f-42c7-410f-8c5d-d436a41b9f4c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:51.942746 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe912d8f-42c7-410f-8c5d-d436a41b9f4c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:51.943178 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe912d8f-42c7-410f-8c5d-d436a41b9f4c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:51.943861 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe912d8f-42c7-410f-8c5d-d436a41b9f4c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:51.944336 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe912d8f-42c7-410f-8c5d-d436a41b9f4c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:51.944759 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe912d8f-42c7-410f-8c5d-d436a41b9f4c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:51.949246 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=100 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:26:52.094756 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fe912d8f-42c7-410f-8c5d-d436a41b9f4c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:52.102580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fe912d8f-42c7-410f-8c5d-d436a41b9f4c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.171278 Aug 21 01:26:52.103145 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 117/239] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:51 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:53.122902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-1391a7c7-7a9d-4554-92b1-ff40d3f10bab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:53.123650 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-1391a7c7-7a9d-4554-92b1-ff40d3f10bab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:53.130730 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1391a7c7-7a9d-4554-92b1-ff40d3f10bab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:53.130730 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1391a7c7-7a9d-4554-92b1-ff40d3f10bab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:53.130730 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1391a7c7-7a9d-4554-92b1-ff40d3f10bab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:53.131252 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1391a7c7-7a9d-4554-92b1-ff40d3f10bab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:53.131702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1391a7c7-7a9d-4554-92b1-ff40d3f10bab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:53.132149 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1391a7c7-7a9d-4554-92b1-ff40d3f10bab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:53.265961 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-1391a7c7-7a9d-4554-92b1-ff40d3f10bab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:53.275659 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-1391a7c7-7a9d-4554-92b1-ff40d3f10bab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.154876 Aug 21 01:26:53.276049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 123/240] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:53 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:54.296612 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-caddcecb-a85a-4371-82f0-1357bca2a1bc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:54.297223 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-caddcecb-a85a-4371-82f0-1357bca2a1bc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:54.303713 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-caddcecb-a85a-4371-82f0-1357bca2a1bc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:54.304224 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-caddcecb-a85a-4371-82f0-1357bca2a1bc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:54.304674 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-caddcecb-a85a-4371-82f0-1357bca2a1bc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:54.306046 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-caddcecb-a85a-4371-82f0-1357bca2a1bc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:54.306527 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-caddcecb-a85a-4371-82f0-1357bca2a1bc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:54.306969 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-caddcecb-a85a-4371-82f0-1357bca2a1bc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:54.490128 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-caddcecb-a85a-4371-82f0-1357bca2a1bc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:54.501266 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-caddcecb-a85a-4371-82f0-1357bca2a1bc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.208450 Aug 21 01:26:54.501842 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 118/241] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:54 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:55.523065 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a6c14989-c9bb-4cb6-8dc0-7b782e2cedb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:55.523655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a6c14989-c9bb-4cb6-8dc0-7b782e2cedb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:55.530424 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a6c14989-c9bb-4cb6-8dc0-7b782e2cedb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:55.530914 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a6c14989-c9bb-4cb6-8dc0-7b782e2cedb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:55.531348 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a6c14989-c9bb-4cb6-8dc0-7b782e2cedb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:55.536853 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a6c14989-c9bb-4cb6-8dc0-7b782e2cedb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:55.536853 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a6c14989-c9bb-4cb6-8dc0-7b782e2cedb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:55.536853 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a6c14989-c9bb-4cb6-8dc0-7b782e2cedb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:55.674864 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a6c14989-c9bb-4cb6-8dc0-7b782e2cedb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:55.683314 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a6c14989-c9bb-4cb6-8dc0-7b782e2cedb1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.163691 Aug 21 01:26:55.683972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 124/242] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:55 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:56.705693 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-361d3e40-7ef4-4a9b-becb-f19ca2e51f40 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:56.706335 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-361d3e40-7ef4-4a9b-becb-f19ca2e51f40 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:56.713293 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-361d3e40-7ef4-4a9b-becb-f19ca2e51f40 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:56.713789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-361d3e40-7ef4-4a9b-becb-f19ca2e51f40 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:56.714255 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-361d3e40-7ef4-4a9b-becb-f19ca2e51f40 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:56.714990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-361d3e40-7ef4-4a9b-becb-f19ca2e51f40 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:56.715577 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-361d3e40-7ef4-4a9b-becb-f19ca2e51f40 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:56.715962 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-361d3e40-7ef4-4a9b-becb-f19ca2e51f40 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:56.849771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-361d3e40-7ef4-4a9b-becb-f19ca2e51f40 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:56.858569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-361d3e40-7ef4-4a9b-becb-f19ca2e51f40 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.156623 Aug 21 01:26:56.859142 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 119/243] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:56 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:57.879757 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6b412f9f-f936-4e0f-9848-c636321f948c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:57.880406 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6b412f9f-f936-4e0f-9848-c636321f948c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:57.887744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6b412f9f-f936-4e0f-9848-c636321f948c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:57.888274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6b412f9f-f936-4e0f-9848-c636321f948c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:57.888714 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6b412f9f-f936-4e0f-9848-c636321f948c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:57.889409 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6b412f9f-f936-4e0f-9848-c636321f948c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:57.889861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6b412f9f-f936-4e0f-9848-c636321f948c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:57.890338 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6b412f9f-f936-4e0f-9848-c636321f948c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:58.019337 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6b412f9f-f936-4e0f-9848-c636321f948c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:58.028878 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6b412f9f-f936-4e0f-9848-c636321f948c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.151399 Aug 21 01:26:58.028878 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 125/244] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:57 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:26:59.049513 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c5522712-4947-453f-ad7d-4e80cdf36c53 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:26:59.050125 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c5522712-4947-453f-ad7d-4e80cdf36c53 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:26:59.056566 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c5522712-4947-453f-ad7d-4e80cdf36c53 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:59.057031 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c5522712-4947-453f-ad7d-4e80cdf36c53 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:59.057468 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c5522712-4947-453f-ad7d-4e80cdf36c53 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:59.058192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c5522712-4947-453f-ad7d-4e80cdf36c53 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:26:59.058644 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c5522712-4947-453f-ad7d-4e80cdf36c53 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:26:59.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c5522712-4947-453f-ad7d-4e80cdf36c53 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:26:59.191016 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c5522712-4947-453f-ad7d-4e80cdf36c53 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:26:59.198574 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c5522712-4947-453f-ad7d-4e80cdf36c53 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.152922 Aug 21 01:26:59.199631 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 120/245] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:26:59 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:00.220556 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-50dccfb4-8ddd-44ae-a2c6-6f32975d5634 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:00.221475 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-50dccfb4-8ddd-44ae-a2c6-6f32975d5634 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:00.227489 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50dccfb4-8ddd-44ae-a2c6-6f32975d5634 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:00.228020 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50dccfb4-8ddd-44ae-a2c6-6f32975d5634 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:00.228455 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50dccfb4-8ddd-44ae-a2c6-6f32975d5634 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:00.229159 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50dccfb4-8ddd-44ae-a2c6-6f32975d5634 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:00.229619 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50dccfb4-8ddd-44ae-a2c6-6f32975d5634 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:00.230045 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50dccfb4-8ddd-44ae-a2c6-6f32975d5634 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:00.382414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-50dccfb4-8ddd-44ae-a2c6-6f32975d5634 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:00.388185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-50dccfb4-8ddd-44ae-a2c6-6f32975d5634 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.171451 Aug 21 01:27:00.388814 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 126/246] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:00 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:01.409439 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9aca192b-1af1-4448-a5b6-d86823594214 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:01.409962 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9aca192b-1af1-4448-a5b6-d86823594214 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:01.412348 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=26 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:01.417335 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9aca192b-1af1-4448-a5b6-d86823594214 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:01.418445 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9aca192b-1af1-4448-a5b6-d86823594214 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:01.418893 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9aca192b-1af1-4448-a5b6-d86823594214 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:01.419569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9aca192b-1af1-4448-a5b6-d86823594214 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:01.420032 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9aca192b-1af1-4448-a5b6-d86823594214 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:01.420468 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9aca192b-1af1-4448-a5b6-d86823594214 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:01.552069 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9aca192b-1af1-4448-a5b6-d86823594214 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:01.560630 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9aca192b-1af1-4448-a5b6-d86823594214 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.155145 Aug 21 01:27:01.561180 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 121/247] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:01 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:02.582343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b3dd1919-b03b-442d-a003-5931738f39cc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:02.582863 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b3dd1919-b03b-442d-a003-5931738f39cc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:02.589231 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b3dd1919-b03b-442d-a003-5931738f39cc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:02.589714 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b3dd1919-b03b-442d-a003-5931738f39cc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:02.590182 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b3dd1919-b03b-442d-a003-5931738f39cc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:02.590871 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b3dd1919-b03b-442d-a003-5931738f39cc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:02.591320 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b3dd1919-b03b-442d-a003-5931738f39cc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:02.591741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b3dd1919-b03b-442d-a003-5931738f39cc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:02.735837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b3dd1919-b03b-442d-a003-5931738f39cc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:02.743825 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b3dd1919-b03b-442d-a003-5931738f39cc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.165382 Aug 21 01:27:02.744749 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 127/248] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:02 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:03.727400 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4ff86580-2fae-4a01-b5db-8381d9146eea tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:03.728061 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4ff86580-2fae-4a01-b5db-8381d9146eea tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:03.739745 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ff86580-2fae-4a01-b5db-8381d9146eea tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:03.740276 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ff86580-2fae-4a01-b5db-8381d9146eea tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:03.740754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ff86580-2fae-4a01-b5db-8381d9146eea tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:03.741521 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ff86580-2fae-4a01-b5db-8381d9146eea tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:03.741977 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ff86580-2fae-4a01-b5db-8381d9146eea tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:03.742407 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ff86580-2fae-4a01-b5db-8381d9146eea tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:03.771258 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c55bf825-ce99-4fc7-8235-7de2b6129370 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:03.771258 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c55bf825-ce99-4fc7-8235-7de2b6129370 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:03.774141 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c55bf825-ce99-4fc7-8235-7de2b6129370 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:03.774638 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c55bf825-ce99-4fc7-8235-7de2b6129370 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:03.775074 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c55bf825-ce99-4fc7-8235-7de2b6129370 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:03.775769 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c55bf825-ce99-4fc7-8235-7de2b6129370 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:03.776254 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c55bf825-ce99-4fc7-8235-7de2b6129370 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:03.776678 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c55bf825-ce99-4fc7-8235-7de2b6129370 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:03.786211 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4ff86580-2fae-4a01-b5db-8381d9146eea tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/ips" status: 200 len: 147 microversion: 2.1 time: 0.065659 Aug 21 01:27:03.787974 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 122/249] 10.222.0.47 () {58 vars in 1248 bytes} [Sun Aug 21 01:27:03 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/ips => generated 147 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:27:03.962838 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c55bf825-ce99-4fc7-8235-7de2b6129370 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:03.971044 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c55bf825-ce99-4fc7-8235-7de2b6129370 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.208646 Aug 21 01:27:03.975235 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 128/250] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:03 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:04.994069 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7586ea95-be01-4c17-af23-80dabe75d14c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:04.994697 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7586ea95-be01-4c17-af23-80dabe75d14c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:05.002055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7586ea95-be01-4c17-af23-80dabe75d14c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:05.002819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7586ea95-be01-4c17-af23-80dabe75d14c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:05.003251 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7586ea95-be01-4c17-af23-80dabe75d14c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:05.003948 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7586ea95-be01-4c17-af23-80dabe75d14c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:05.004663 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7586ea95-be01-4c17-af23-80dabe75d14c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:05.005089 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7586ea95-be01-4c17-af23-80dabe75d14c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:05.139828 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7586ea95-be01-4c17-af23-80dabe75d14c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:05.147949 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7586ea95-be01-4c17-af23-80dabe75d14c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.157802 Aug 21 01:27:05.148538 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 123/251] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:04 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:06.168988 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2bcadb74-86ff-4d9d-87d4-61213a275133 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:06.169581 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-2bcadb74-86ff-4d9d-87d4-61213a275133 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:06.175823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2bcadb74-86ff-4d9d-87d4-61213a275133 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:06.176351 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2bcadb74-86ff-4d9d-87d4-61213a275133 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:06.176789 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2bcadb74-86ff-4d9d-87d4-61213a275133 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:06.177476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2bcadb74-86ff-4d9d-87d4-61213a275133 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:06.177930 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2bcadb74-86ff-4d9d-87d4-61213a275133 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:06.178358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2bcadb74-86ff-4d9d-87d4-61213a275133 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:06.307988 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2bcadb74-86ff-4d9d-87d4-61213a275133 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:06.316714 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2bcadb74-86ff-4d9d-87d4-61213a275133 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.151816 Aug 21 01:27:06.317325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 129/252] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:06 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:07.337776 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c86512e6-5928-4cf0-96ce-9f6e6b007ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:07.338411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c86512e6-5928-4cf0-96ce-9f6e6b007ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:07.346061 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c86512e6-5928-4cf0-96ce-9f6e6b007ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:07.346541 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c86512e6-5928-4cf0-96ce-9f6e6b007ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:07.346974 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c86512e6-5928-4cf0-96ce-9f6e6b007ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:07.347809 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c86512e6-5928-4cf0-96ce-9f6e6b007ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:07.348294 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c86512e6-5928-4cf0-96ce-9f6e6b007ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:07.348714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c86512e6-5928-4cf0-96ce-9f6e6b007ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:07.502558 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c86512e6-5928-4cf0-96ce-9f6e6b007ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:07.510270 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c86512e6-5928-4cf0-96ce-9f6e6b007ee4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.176479 Aug 21 01:27:07.510832 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 124/253] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:07 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:08.532476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5df600ac-6c6c-41e6-98c7-6b88249b575e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:08.533232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5df600ac-6c6c-41e6-98c7-6b88249b575e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:08.542475 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5df600ac-6c6c-41e6-98c7-6b88249b575e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:08.542475 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5df600ac-6c6c-41e6-98c7-6b88249b575e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:08.542475 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5df600ac-6c6c-41e6-98c7-6b88249b575e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:08.542475 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5df600ac-6c6c-41e6-98c7-6b88249b575e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:08.542844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5df600ac-6c6c-41e6-98c7-6b88249b575e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:08.542844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5df600ac-6c6c-41e6-98c7-6b88249b575e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:08.670234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5df600ac-6c6c-41e6-98c7-6b88249b575e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:08.678256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5df600ac-6c6c-41e6-98c7-6b88249b575e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.150320 Aug 21 01:27:08.678860 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 130/254] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:08 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:09.699881 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c1524a7c-48c9-421f-af78-7ae26e71c76d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:09.700533 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c1524a7c-48c9-421f-af78-7ae26e71c76d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:09.707736 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c1524a7c-48c9-421f-af78-7ae26e71c76d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:09.708254 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c1524a7c-48c9-421f-af78-7ae26e71c76d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:09.708695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c1524a7c-48c9-421f-af78-7ae26e71c76d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:09.709380 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c1524a7c-48c9-421f-af78-7ae26e71c76d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:09.709832 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c1524a7c-48c9-421f-af78-7ae26e71c76d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:09.710254 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c1524a7c-48c9-421f-af78-7ae26e71c76d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:09.850591 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c1524a7c-48c9-421f-af78-7ae26e71c76d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:09.870805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c1524a7c-48c9-421f-af78-7ae26e71c76d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.166628 Aug 21 01:27:09.871798 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 125/255] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:09 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:10.883146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d54e2125-7689-4611-9819-8a53f42609e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:10.883748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d54e2125-7689-4611-9819-8a53f42609e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:10.892241 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d54e2125-7689-4611-9819-8a53f42609e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:10.892761 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d54e2125-7689-4611-9819-8a53f42609e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:10.893379 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d54e2125-7689-4611-9819-8a53f42609e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:10.894078 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d54e2125-7689-4611-9819-8a53f42609e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:10.894534 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d54e2125-7689-4611-9819-8a53f42609e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:10.894954 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d54e2125-7689-4611-9819-8a53f42609e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:11.036085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d54e2125-7689-4611-9819-8a53f42609e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:11.044486 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d54e2125-7689-4611-9819-8a53f42609e7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.165338 Aug 21 01:27:11.045069 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 131/256] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:10 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:12.066845 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-10f0e5f7-ae0f-490b-8912-5fa03b008b9b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:12.067454 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-10f0e5f7-ae0f-490b-8912-5fa03b008b9b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:12.074161 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-10f0e5f7-ae0f-490b-8912-5fa03b008b9b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:12.074675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-10f0e5f7-ae0f-490b-8912-5fa03b008b9b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:12.075113 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-10f0e5f7-ae0f-490b-8912-5fa03b008b9b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:12.075812 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-10f0e5f7-ae0f-490b-8912-5fa03b008b9b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:12.076302 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-10f0e5f7-ae0f-490b-8912-5fa03b008b9b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:12.076738 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-10f0e5f7-ae0f-490b-8912-5fa03b008b9b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:12.210276 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-10f0e5f7-ae0f-490b-8912-5fa03b008b9b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:12.221037 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-10f0e5f7-ae0f-490b-8912-5fa03b008b9b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.157839 Aug 21 01:27:12.222158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 126/257] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:12 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:13.209619 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d932cf07-a017-4bc6-b502-0d4b0f9f936e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bb48bf98-8498-4b4f-bd2a-cbfdc1713fed"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:13.212545 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d932cf07-a017-4bc6-b502-0d4b0f9f936e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:13.219371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d932cf07-a017-4bc6-b502-0d4b0f9f936e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:13.222096 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d932cf07-a017-4bc6-b502-0d4b0f9f936e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:13.222096 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d932cf07-a017-4bc6-b502-0d4b0f9f936e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:13.222096 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d932cf07-a017-4bc6-b502-0d4b0f9f936e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:13.222096 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d932cf07-a017-4bc6-b502-0d4b0f9f936e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:13.223008 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d932cf07-a017-4bc6-b502-0d4b0f9f936e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:13.246400 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6c417eb8-502e-4641-b285-2c8331cd817b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:13.247013 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6c417eb8-502e-4641-b285-2c8331cd817b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:13.257915 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c417eb8-502e-4641-b285-2c8331cd817b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:13.258797 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c417eb8-502e-4641-b285-2c8331cd817b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:13.259304 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c417eb8-502e-4641-b285-2c8331cd817b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:13.260178 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c417eb8-502e-4641-b285-2c8331cd817b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:13.260943 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c417eb8-502e-4641-b285-2c8331cd817b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:13.261325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c417eb8-502e-4641-b285-2c8331cd817b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:13.430378 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6c417eb8-502e-4641-b285-2c8331cd817b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:13.441267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6c417eb8-502e-4641-b285-2c8331cd817b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.198682 Aug 21 01:27:13.442056 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 127/258] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:13 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:14.463333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-10e1e464-2934-4a35-8af2-edc5f8ec0a33 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:14.464042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-10e1e464-2934-4a35-8af2-edc5f8ec0a33 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:14.471413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-10e1e464-2934-4a35-8af2-edc5f8ec0a33 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:14.472311 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-10e1e464-2934-4a35-8af2-edc5f8ec0a33 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:14.472811 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-10e1e464-2934-4a35-8af2-edc5f8ec0a33 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:14.473584 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-10e1e464-2934-4a35-8af2-edc5f8ec0a33 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:14.474106 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-10e1e464-2934-4a35-8af2-edc5f8ec0a33 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:14.474596 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-10e1e464-2934-4a35-8af2-edc5f8ec0a33 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:14.618038 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-10e1e464-2934-4a35-8af2-edc5f8ec0a33 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:14.629464 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-10e1e464-2934-4a35-8af2-edc5f8ec0a33 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.170852 Aug 21 01:27:14.630054 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 128/259] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:14 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:27:14.718461 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d932cf07-a017-4bc6-b502-0d4b0f9f936e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "POST /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.513232 Aug 21 01:27:14.719071 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 132/260] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:27:13 2022] POST /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/os-volume_attachments => generated 194 bytes in 1515 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:27:14.932030 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9fbb6643-f968-4124-a2fa-0f82d586fffa tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:14.932645 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9fbb6643-f968-4124-a2fa-0f82d586fffa tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:14.939616 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9fbb6643-f968-4124-a2fa-0f82d586fffa tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:14.940095 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9fbb6643-f968-4124-a2fa-0f82d586fffa tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:14.940550 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9fbb6643-f968-4124-a2fa-0f82d586fffa tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:14.941238 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9fbb6643-f968-4124-a2fa-0f82d586fffa tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:14.941687 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9fbb6643-f968-4124-a2fa-0f82d586fffa tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:14.942106 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9fbb6643-f968-4124-a2fa-0f82d586fffa tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:14.967005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9fbb6643-f968-4124-a2fa-0f82d586fffa tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/ips" status: 200 len: 151 microversion: 2.20 time: 0.038933 Aug 21 01:27:14.967793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 129/261] 10.222.0.47 () {60 vars in 1289 bytes} [Sun Aug 21 01:27:14 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/ips => generated 151 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:27:15.652254 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-043c8bb8-e6db-4873-b91c-dc96066f3e2d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:15.653139 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-043c8bb8-e6db-4873-b91c-dc96066f3e2d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:15.655092 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=32 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:15.660406 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-043c8bb8-e6db-4873-b91c-dc96066f3e2d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:15.660893 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-043c8bb8-e6db-4873-b91c-dc96066f3e2d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:15.661381 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-043c8bb8-e6db-4873-b91c-dc96066f3e2d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:15.662067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-043c8bb8-e6db-4873-b91c-dc96066f3e2d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:15.662517 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-043c8bb8-e6db-4873-b91c-dc96066f3e2d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:15.662936 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-043c8bb8-e6db-4873-b91c-dc96066f3e2d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:15.684422 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:15.832558 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-043c8bb8-e6db-4873-b91c-dc96066f3e2d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:15.844901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-043c8bb8-e6db-4873-b91c-dc96066f3e2d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.196841 Aug 21 01:27:15.848648 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 133/262] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:15 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:16.867822 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-54b50f1b-5373-4dcb-9118-bbf0b18d0f36 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:16.868490 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-54b50f1b-5373-4dcb-9118-bbf0b18d0f36 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:16.875047 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-54b50f1b-5373-4dcb-9118-bbf0b18d0f36 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:16.876054 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-54b50f1b-5373-4dcb-9118-bbf0b18d0f36 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:16.876530 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-54b50f1b-5373-4dcb-9118-bbf0b18d0f36 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:16.877222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-54b50f1b-5373-4dcb-9118-bbf0b18d0f36 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:16.877686 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-54b50f1b-5373-4dcb-9118-bbf0b18d0f36 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:16.878301 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-54b50f1b-5373-4dcb-9118-bbf0b18d0f36 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:17.028535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-54b50f1b-5373-4dcb-9118-bbf0b18d0f36 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:17.032079 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-54b50f1b-5373-4dcb-9118-bbf0b18d0f36 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1502 microversion: 2.1 time: 0.168069 Aug 21 01:27:17.032710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 130/263] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:16 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1502 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:17.053428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-268f4546-2661-48ec-92f6-a3acc6eb99b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8567e94e-6325-48aa-96a7-7666c82d879f"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:17.055868 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-268f4546-2661-48ec-92f6-a3acc6eb99b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:17.063782 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-268f4546-2661-48ec-92f6-a3acc6eb99b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:17.064335 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-268f4546-2661-48ec-92f6-a3acc6eb99b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:17.064826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-268f4546-2661-48ec-92f6-a3acc6eb99b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:17.065646 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-268f4546-2661-48ec-92f6-a3acc6eb99b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:17.066148 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-268f4546-2661-48ec-92f6-a3acc6eb99b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:17.066616 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-268f4546-2661-48ec-92f6-a3acc6eb99b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:17.093906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-268f4546-2661-48ec-92f6-a3acc6eb99b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] HTTP exception thrown: Cannot 'attach_volume' instance 90e88110-1175-4ccf-bf6b-8efb43ab945e while it is in vm_state rescued Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-268f4546-2661-48ec-92f6-a3acc6eb99b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Request method failure captured: Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: POST /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e/os-volume_attachments HTTP/1.1 Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 74 Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYl2grkk6r6DOR4WcdCDQaZ1LKSwt5ty8EvPKEsirHl25Bbz12fukVsuTZFRPo5OnAmJDHzJfldGu7OE-fBThLL0sUCHPYzsZbzZPojBGa0s89nUPxFfxXEogZCrYm2ZzOthYXvM-jM7ah9S1dz6CN84xJGSagaAUIMSBiUILa4u8mo Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 9774c354b8b048768f2071acfb48a4bf Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:27:17.096285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:27:17.097857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:27:17.097857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:27:17.097857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 9774c354b8b048768f2071acfb48a4bf Aug 21 01:27:17.097857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:27:17.097857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-ServerRescueNegativeTestJSON-1204886246-project Aug 21 01:27:17.097857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:27:17.097857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:27:17.097857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: d84b6d05decb47c7b55ac782d5839c1a Aug 21 01:27:17.097857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1204886246-project Aug 21 01:27:17.097857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Aug 21 01:27:17.097857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {"volumeAttachment": {"volumeId": "8567e94e-6325-48aa-96a7-7666c82d879f"}} Aug 21 01:27:17.097857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Cannot 'attach_volume' instance 90e88110-1175-4ccf-bf6b-8efb43ab945e while it is in vm_state rescued Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'server_id': '90e88110-1175-4ccf-bf6b-8efb43ab945e', 'body': {'volumeAttachment': {'volumeId': '8567e94e-6325-48aa-96a7-7666c82d879f'}}} Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 90e88110-1175-4ccf-bf6b-8efb43ab945e in vm_state rescued. Cannot attach_volume while the instance is in this state. Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:27:17.098969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 90e88110-1175-4ccf-bf6b-8efb43ab945e while it is in vm_state rescued Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:27:17.103824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:27:17.105468 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:27:17.105468 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:27:17.105468 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 90e88110-1175-4ccf-bf6b-8efb43ab945e while it is in vm_state rescued Aug 21 01:27:17.105468 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:27:17.105468 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-268f4546-2661-48ec-92f6-a3acc6eb99b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Returning 409 to user: Cannot 'attach_volume' instance 90e88110-1175-4ccf-bf6b-8efb43ab945e while it is in vm_state rescued {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:27:17.105468 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-268f4546-2661-48ec-92f6-a3acc6eb99b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "POST /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.048532 Aug 21 01:27:17.105468 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 134/264] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:27:17 2022] POST /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e/os-volume_attachments => generated 152 bytes in 51 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Aug 21 01:27:17.111344 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-905a0402-ec1d-411f-95c1-78e77657c017 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:17.111959 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-905a0402-ec1d-411f-95c1-78e77657c017 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:17.118137 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-905a0402-ec1d-411f-95c1-78e77657c017 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:17.118813 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-905a0402-ec1d-411f-95c1-78e77657c017 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:17.119418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-905a0402-ec1d-411f-95c1-78e77657c017 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:17.120405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-905a0402-ec1d-411f-95c1-78e77657c017 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:17.121070 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-905a0402-ec1d-411f-95c1-78e77657c017 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:17.121687 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-905a0402-ec1d-411f-95c1-78e77657c017 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:17.445408 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-905a0402-ec1d-411f-95c1-78e77657c017 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "POST /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e/action" status: 202 len: 0 microversion: 2.1 time: 0.337797 Aug 21 01:27:17.449094 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 131/265] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:27:17 2022] POST /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e/action => generated 0 bytes in 340 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:27:17.455365 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-32e8a1d1-fab5-40c1-b217-234d02278b60 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:17.456145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-32e8a1d1-fab5-40c1-b217-234d02278b60 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:17.463467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-32e8a1d1-fab5-40c1-b217-234d02278b60 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:17.463958 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-32e8a1d1-fab5-40c1-b217-234d02278b60 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:17.464810 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-32e8a1d1-fab5-40c1-b217-234d02278b60 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:17.465425 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-32e8a1d1-fab5-40c1-b217-234d02278b60 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:17.465880 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-32e8a1d1-fab5-40c1-b217-234d02278b60 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:17.466303 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-32e8a1d1-fab5-40c1-b217-234d02278b60 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:17.607445 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-32e8a1d1-fab5-40c1-b217-234d02278b60 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:17.618115 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-32e8a1d1-fab5-40c1-b217-234d02278b60 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1510 microversion: 2.1 time: 0.166907 Aug 21 01:27:17.622729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 135/266] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:17 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1510 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:17.781666 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-db378a11-533d-4cd8-a3c4-ff4e9dc8603e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:17.783141 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-db378a11-533d-4cd8-a3c4-ff4e9dc8603e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:17.789763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db378a11-533d-4cd8-a3c4-ff4e9dc8603e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:17.790474 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db378a11-533d-4cd8-a3c4-ff4e9dc8603e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:17.791136 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db378a11-533d-4cd8-a3c4-ff4e9dc8603e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:17.791910 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db378a11-533d-4cd8-a3c4-ff4e9dc8603e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:17.792823 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db378a11-533d-4cd8-a3c4-ff4e9dc8603e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:17.793475 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db378a11-533d-4cd8-a3c4-ff4e9dc8603e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:17.819041 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-db378a11-533d-4cd8-a3c4-ff4e9dc8603e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lazy-loading 'flavor' on Instance uuid 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:27:18.026584 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:18.642371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-1f3934a4-19da-4bdf-be29-452125ad05ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:18.643233 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-1f3934a4-19da-4bdf-be29-452125ad05ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:18.651883 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1f3934a4-19da-4bdf-be29-452125ad05ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:18.652553 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1f3934a4-19da-4bdf-be29-452125ad05ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:18.653374 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1f3934a4-19da-4bdf-be29-452125ad05ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:18.654611 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1f3934a4-19da-4bdf-be29-452125ad05ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:18.655157 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1f3934a4-19da-4bdf-be29-452125ad05ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:18.655657 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1f3934a4-19da-4bdf-be29-452125ad05ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:18.685244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-db378a11-533d-4cd8-a3c4-ff4e9dc8603e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "POST /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/action" status: 202 len: 0 microversion: 2.20 time: 0.907764 Aug 21 01:27:18.686456 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 132/267] 10.222.0.47 () {62 vars in 1319 bytes} [Sun Aug 21 01:27:17 2022] POST /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/action => generated 0 bytes in 910 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:27:18.700228 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-60e5a8ff-b36c-4f38-87c0-1433a23e4387 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:18.701700 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-60e5a8ff-b36c-4f38-87c0-1433a23e4387 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:18.711973 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-60e5a8ff-b36c-4f38-87c0-1433a23e4387 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:18.713425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-60e5a8ff-b36c-4f38-87c0-1433a23e4387 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:18.714136 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-60e5a8ff-b36c-4f38-87c0-1433a23e4387 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:18.715111 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-60e5a8ff-b36c-4f38-87c0-1433a23e4387 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:18.715828 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-60e5a8ff-b36c-4f38-87c0-1433a23e4387 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:18.716587 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-60e5a8ff-b36c-4f38-87c0-1433a23e4387 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:18.894121 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-1f3934a4-19da-4bdf-be29-452125ad05ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:18.911440 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-1f3934a4-19da-4bdf-be29-452125ad05ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1510 microversion: 2.1 time: 0.271720 Aug 21 01:27:18.920064 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 136/268] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:18 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1510 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:18.955985 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-60e5a8ff-b36c-4f38-87c0-1433a23e4387 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:18.964843 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-60e5a8ff-b36c-4f38-87c0-1433a23e4387 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:18.966356 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-60e5a8ff-b36c-4f38-87c0-1433a23e4387 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:18.967561 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-60e5a8ff-b36c-4f38-87c0-1433a23e4387 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.273374 Aug 21 01:27:18.968208 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 133/269] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:18 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:19.934344 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ae34076a-ebc7-4b91-a5ba-a4d591bdc4d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:19.934955 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-ae34076a-ebc7-4b91-a5ba-a4d591bdc4d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:19.942308 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ae34076a-ebc7-4b91-a5ba-a4d591bdc4d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:19.942742 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ae34076a-ebc7-4b91-a5ba-a4d591bdc4d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:19.943182 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ae34076a-ebc7-4b91-a5ba-a4d591bdc4d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:19.946304 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ae34076a-ebc7-4b91-a5ba-a4d591bdc4d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:19.946304 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ae34076a-ebc7-4b91-a5ba-a4d591bdc4d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:19.946304 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ae34076a-ebc7-4b91-a5ba-a4d591bdc4d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:19.995349 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8053fb1d-f59e-422d-8fbc-7a70cd5029d2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:19.996024 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8053fb1d-f59e-422d-8fbc-7a70cd5029d2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:20.004708 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8053fb1d-f59e-422d-8fbc-7a70cd5029d2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:20.005484 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8053fb1d-f59e-422d-8fbc-7a70cd5029d2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:20.006062 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8053fb1d-f59e-422d-8fbc-7a70cd5029d2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:20.006962 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8053fb1d-f59e-422d-8fbc-7a70cd5029d2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:20.007745 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8053fb1d-f59e-422d-8fbc-7a70cd5029d2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:20.008595 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8053fb1d-f59e-422d-8fbc-7a70cd5029d2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:20.224961 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8053fb1d-f59e-422d-8fbc-7a70cd5029d2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:20.233692 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8053fb1d-f59e-422d-8fbc-7a70cd5029d2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:20.235199 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8053fb1d-f59e-422d-8fbc-7a70cd5029d2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:20.237225 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8053fb1d-f59e-422d-8fbc-7a70cd5029d2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.249708 Aug 21 01:27:20.237939 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 134/270] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:19 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:27:20.293856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ae34076a-ebc7-4b91-a5ba-a4d591bdc4d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:20.303137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ae34076a-ebc7-4b91-a5ba-a4d591bdc4d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1510 microversion: 2.1 time: 0.372632 Aug 21 01:27:20.303878 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 137/271] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:19 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1510 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:21.261045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-0bb24a15-d080-41b5-8a96-f27187d72221 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:21.261652 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-0bb24a15-d080-41b5-8a96-f27187d72221 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:21.268807 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0bb24a15-d080-41b5-8a96-f27187d72221 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:21.269489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0bb24a15-d080-41b5-8a96-f27187d72221 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:21.269691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0bb24a15-d080-41b5-8a96-f27187d72221 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:21.270446 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0bb24a15-d080-41b5-8a96-f27187d72221 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:21.271330 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0bb24a15-d080-41b5-8a96-f27187d72221 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:21.271754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0bb24a15-d080-41b5-8a96-f27187d72221 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:21.330005 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-0524fa7f-1906-4388-b94e-8feb6d7996bb tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:21.331987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-0524fa7f-1906-4388-b94e-8feb6d7996bb tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:21.342292 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0524fa7f-1906-4388-b94e-8feb6d7996bb tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:21.343027 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0524fa7f-1906-4388-b94e-8feb6d7996bb tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:21.344203 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0524fa7f-1906-4388-b94e-8feb6d7996bb tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:21.345465 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0524fa7f-1906-4388-b94e-8feb6d7996bb tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:21.346367 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0524fa7f-1906-4388-b94e-8feb6d7996bb tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:21.348299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0524fa7f-1906-4388-b94e-8feb6d7996bb tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:21.474033 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0bb24a15-d080-41b5-8a96-f27187d72221 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:21.484944 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0bb24a15-d080-41b5-8a96-f27187d72221 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:21.487065 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0bb24a15-d080-41b5-8a96-f27187d72221 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:21.488804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-0bb24a15-d080-41b5-8a96-f27187d72221 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.231309 Aug 21 01:27:21.489544 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 135/272] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:21 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:21.523324 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-0524fa7f-1906-4388-b94e-8feb6d7996bb tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:21.540215 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-0524fa7f-1906-4388-b94e-8feb6d7996bb tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1516 microversion: 2.1 time: 0.209961 Aug 21 01:27:21.540215 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 138/273] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:21 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1516 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:27:22.511438 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f1b7d3ed-0253-4b84-85ad-df2d9d57db70 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:22.512184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f1b7d3ed-0253-4b84-85ad-df2d9d57db70 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:22.519201 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1b7d3ed-0253-4b84-85ad-df2d9d57db70 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:22.519691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1b7d3ed-0253-4b84-85ad-df2d9d57db70 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:22.520158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1b7d3ed-0253-4b84-85ad-df2d9d57db70 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:22.520849 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1b7d3ed-0253-4b84-85ad-df2d9d57db70 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:22.521323 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1b7d3ed-0253-4b84-85ad-df2d9d57db70 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:22.521778 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1b7d3ed-0253-4b84-85ad-df2d9d57db70 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:22.843191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f1b7d3ed-0253-4b84-85ad-df2d9d57db70 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:22.851978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f1b7d3ed-0253-4b84-85ad-df2d9d57db70 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:22.853612 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f1b7d3ed-0253-4b84-85ad-df2d9d57db70 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:22.854970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f1b7d3ed-0253-4b84-85ad-df2d9d57db70 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.347559 Aug 21 01:27:22.855524 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 136/274] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:22 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:27:23.262716 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-14fe5eb8-ee93-4a36-9689-9e338e1c6c07 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Action: 'create', calling method: .version_select at 0x7f9da8045a60>, body: {"keypair": {"name": "tempest-keypair-980024713", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOmrMwbc+p6/hfREiiI2KF1mKEO2m+FJ4WK9dcWWuNTdUBsKT11db0ygo0unvWMYVmw2SHlQ88tE5yjAykNMwPQLaMv7WTADTtMV7XBMjQCb8WbCMZN3viFxGtybYVaINA=="}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:23.278656 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-14fe5eb8-ee93-4a36-9689-9e338e1c6c07 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Getting quotas for project 9774c354b8b048768f2071acfb48a4bf. Resources: {'key_pairs'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:27:23.282444 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-14fe5eb8-ee93-4a36-9689-9e338e1c6c07 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Getting quotas for user d84b6d05decb47c7b55ac782d5839c1a and project 9774c354b8b048768f2071acfb48a4bf. Resources: {'key_pairs'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:27:23.381385 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-14fe5eb8-ee93-4a36-9689-9e338e1c6c07 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.122364 Aug 21 01:27:23.382138 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 139/275] 10.222.0.47 () {60 vars in 1159 bytes} [Sun Aug 21 01:27:23 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:27:23.878925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-927391bf-71fa-4b7c-85ba-9c3012be0423 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:23.880587 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-927391bf-71fa-4b7c-85ba-9c3012be0423 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:23.890044 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-927391bf-71fa-4b7c-85ba-9c3012be0423 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:23.891332 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-927391bf-71fa-4b7c-85ba-9c3012be0423 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:23.892048 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-927391bf-71fa-4b7c-85ba-9c3012be0423 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:23.894119 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-927391bf-71fa-4b7c-85ba-9c3012be0423 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:23.894932 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-927391bf-71fa-4b7c-85ba-9c3012be0423 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:23.895528 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-927391bf-71fa-4b7c-85ba-9c3012be0423 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:24.039337 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-927391bf-71fa-4b7c-85ba-9c3012be0423 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:24.053195 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-927391bf-71fa-4b7c-85ba-9c3012be0423 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:24.055271 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-927391bf-71fa-4b7c-85ba-9c3012be0423 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:24.056733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-927391bf-71fa-4b7c-85ba-9c3012be0423 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.182785 Aug 21 01:27:24.057781 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 137/276] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:23 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:25.080941 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-143b8d9c-e7a5-4ddf-bfb3-452de0ff5926 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:25.082203 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-143b8d9c-e7a5-4ddf-bfb3-452de0ff5926 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:25.100342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-143b8d9c-e7a5-4ddf-bfb3-452de0ff5926 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:25.100342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-143b8d9c-e7a5-4ddf-bfb3-452de0ff5926 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:25.100342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-143b8d9c-e7a5-4ddf-bfb3-452de0ff5926 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:25.100342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-143b8d9c-e7a5-4ddf-bfb3-452de0ff5926 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:25.100745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-143b8d9c-e7a5-4ddf-bfb3-452de0ff5926 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:25.100745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-143b8d9c-e7a5-4ddf-bfb3-452de0ff5926 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:25.183858 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4a860164-81fd-49da-889b-52720a3278a4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:25.195912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4a860164-81fd-49da-889b-52720a3278a4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:25.196585 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4a860164-81fd-49da-889b-52720a3278a4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:25.197127 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4a860164-81fd-49da-889b-52720a3278a4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:25.318911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4a860164-81fd-49da-889b-52720a3278a4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Going to try to stop instance {{(pid=121276) force_stop /opt/stack/nova/nova/compute/api.py:2749}} Aug 21 01:27:25.607543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-143b8d9c-e7a5-4ddf-bfb3-452de0ff5926 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:25.610734 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4a860164-81fd-49da-889b-52720a3278a4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "POST /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/action" status: 202 len: 0 microversion: 2.1 time: 0.428212 Aug 21 01:27:25.613972 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 138/277] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:27:25 2022] POST /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/action => generated 0 bytes in 435 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 21 01:27:25.638035 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-143b8d9c-e7a5-4ddf-bfb3-452de0ff5926 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:25.638537 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-576db78d-6c1e-4323-935b-1b172f531756 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:25.641714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-576db78d-6c1e-4323-935b-1b172f531756 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:25.647310 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-143b8d9c-e7a5-4ddf-bfb3-452de0ff5926 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:25.655013 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-143b8d9c-e7a5-4ddf-bfb3-452de0ff5926 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.572387 Aug 21 01:27:25.675741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 140/278] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:25 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:27:25.676378 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-576db78d-6c1e-4323-935b-1b172f531756 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:25.678816 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-576db78d-6c1e-4323-935b-1b172f531756 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:25.679326 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-576db78d-6c1e-4323-935b-1b172f531756 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:25.680037 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-576db78d-6c1e-4323-935b-1b172f531756 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:25.680557 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-576db78d-6c1e-4323-935b-1b172f531756 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:25.680988 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-576db78d-6c1e-4323-935b-1b172f531756 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:26.000594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-576db78d-6c1e-4323-935b-1b172f531756 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:26.009799 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-576db78d-6c1e-4323-935b-1b172f531756 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.389635 Aug 21 01:27:26.024340 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 139/279] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:25 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:26.686823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2c2f646d-698e-4355-8a05-a2da65474fd7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:26.686823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-2c2f646d-698e-4355-8a05-a2da65474fd7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:26.693655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2c2f646d-698e-4355-8a05-a2da65474fd7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:26.694409 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2c2f646d-698e-4355-8a05-a2da65474fd7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:26.695040 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2c2f646d-698e-4355-8a05-a2da65474fd7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:26.696057 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2c2f646d-698e-4355-8a05-a2da65474fd7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:26.696905 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2c2f646d-698e-4355-8a05-a2da65474fd7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:26.697482 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2c2f646d-698e-4355-8a05-a2da65474fd7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:26.819293 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-315277938", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "679614c9-0d32-43d1-9058-1d318da99722"}], "security_groups": [{"name": "tempest-securitygroup--1339996228"}], "key_name": "tempest-keypair-980024713", "user_data": "}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:27.167913 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] validate_networks() for [('679614c9-0d32-43d1-9058-1d318da99722', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:27:27.295853 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2c2f646d-698e-4355-8a05-a2da65474fd7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:27.307239 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2c2f646d-698e-4355-8a05-a2da65474fd7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:27.309481 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2c2f646d-698e-4355-8a05-a2da65474fd7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:27.310659 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2c2f646d-698e-4355-8a05-a2da65474fd7 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.629551 Aug 21 01:27:27.318239 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 141/280] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:26 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:27.324652 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f2bac751-409c-4e65-a8d9-c21daabab0ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:27.325693 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f2bac751-409c-4e65-a8d9-c21daabab0ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:27.334250 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2bac751-409c-4e65-a8d9-c21daabab0ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:27.338112 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2bac751-409c-4e65-a8d9-c21daabab0ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:27.338112 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2bac751-409c-4e65-a8d9-c21daabab0ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:27.338112 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2bac751-409c-4e65-a8d9-c21daabab0ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:27.338112 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2bac751-409c-4e65-a8d9-c21daabab0ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:27.338589 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2bac751-409c-4e65-a8d9-c21daabab0ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:27.671551 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f2bac751-409c-4e65-a8d9-c21daabab0ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:27.682282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f2bac751-409c-4e65-a8d9-c21daabab0ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.361378 Aug 21 01:27:27.682895 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 142/281] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:27 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:27.985236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:27:27.985236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:27:27.986885 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:27:27.987417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:27:27.988241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:27:28.001744 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:28.003161 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:28.004091 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:28.005032 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:28.005756 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:28.006415 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:28.027218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Getting quotas for project 9774c354b8b048768f2071acfb48a4bf. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:27:28.032450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Getting quotas for user d84b6d05decb47c7b55ac782d5839c1a and project 9774c354b8b048768f2071acfb48a4bf. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:27:28.049268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:27:28.052777 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:27:28.191711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-05c519df-ed49-43a1-a92a-b393e0a0656f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.377479 Aug 21 01:27:28.192641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 140/282] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:27:26 2022] POST /compute/v2.1/servers => generated 409 bytes in 1379 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 21 01:27:28.208432 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-45ef32a2-74ac-4e73-ba29-31658bf4a8f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:28.211076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-45ef32a2-74ac-4e73-ba29-31658bf4a8f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:28.336817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f5de0a20-8dd3-4687-9495-749b201c88d5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:28.336817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f5de0a20-8dd3-4687-9495-749b201c88d5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:28.356435 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f5de0a20-8dd3-4687-9495-749b201c88d5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:28.356936 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f5de0a20-8dd3-4687-9495-749b201c88d5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:28.357430 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f5de0a20-8dd3-4687-9495-749b201c88d5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:28.358084 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f5de0a20-8dd3-4687-9495-749b201c88d5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:28.358554 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f5de0a20-8dd3-4687-9495-749b201c88d5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:28.358945 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f5de0a20-8dd3-4687-9495-749b201c88d5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:28.448983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-45ef32a2-74ac-4e73-ba29-31658bf4a8f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:28.462638 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-45ef32a2-74ac-4e73-ba29-31658bf4a8f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1244 microversion: 2.1 time: 0.257747 Aug 21 01:27:28.462638 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 143/283] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:28 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1244 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:28.680159 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f5de0a20-8dd3-4687-9495-749b201c88d5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:28.692514 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f5de0a20-8dd3-4687-9495-749b201c88d5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:28.697833 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f5de0a20-8dd3-4687-9495-749b201c88d5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:28.697833 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f5de0a20-8dd3-4687-9495-749b201c88d5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.365596 Aug 21 01:27:28.707750 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-0f4ce86f-4efe-4893-878f-c5521c135a0b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:28.708523 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-0f4ce86f-4efe-4893-878f-c5521c135a0b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:28.713325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 141/284] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:28 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:28.719761 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0f4ce86f-4efe-4893-878f-c5521c135a0b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:28.720415 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0f4ce86f-4efe-4893-878f-c5521c135a0b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:28.720865 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0f4ce86f-4efe-4893-878f-c5521c135a0b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:28.721946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0f4ce86f-4efe-4893-878f-c5521c135a0b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:28.722505 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0f4ce86f-4efe-4893-878f-c5521c135a0b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:28.722943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0f4ce86f-4efe-4893-878f-c5521c135a0b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:28.940540 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-0f4ce86f-4efe-4893-878f-c5521c135a0b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:28.951069 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-0f4ce86f-4efe-4893-878f-c5521c135a0b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.251799 Aug 21 01:27:28.951993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 144/285] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:28 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:29.485975 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-24f688e7-591b-43c8-83de-7b5d827b7830 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:29.487436 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-24f688e7-591b-43c8-83de-7b5d827b7830 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:29.495393 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-24f688e7-591b-43c8-83de-7b5d827b7830 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:29.496347 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-24f688e7-591b-43c8-83de-7b5d827b7830 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:29.497007 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-24f688e7-591b-43c8-83de-7b5d827b7830 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:29.498047 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-24f688e7-591b-43c8-83de-7b5d827b7830 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:29.498569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-24f688e7-591b-43c8-83de-7b5d827b7830 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:29.499305 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-24f688e7-591b-43c8-83de-7b5d827b7830 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:29.575657 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-24f688e7-591b-43c8-83de-7b5d827b7830 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:29.586752 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-24f688e7-591b-43c8-83de-7b5d827b7830 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1248 microversion: 2.1 time: 0.105560 Aug 21 01:27:29.587330 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 142/286] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:29 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1248 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:29.722196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f2bc4fca-6f48-4308-ab9b-fdca7ee2d9af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:29.722811 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f2bc4fca-6f48-4308-ab9b-fdca7ee2d9af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:29.730295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2bc4fca-6f48-4308-ab9b-fdca7ee2d9af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:29.730794 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2bc4fca-6f48-4308-ab9b-fdca7ee2d9af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:29.731245 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2bc4fca-6f48-4308-ab9b-fdca7ee2d9af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:29.731974 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2bc4fca-6f48-4308-ab9b-fdca7ee2d9af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:29.732531 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2bc4fca-6f48-4308-ab9b-fdca7ee2d9af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:29.732965 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f2bc4fca-6f48-4308-ab9b-fdca7ee2d9af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:29.910725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f2bc4fca-6f48-4308-ab9b-fdca7ee2d9af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:29.921648 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f2bc4fca-6f48-4308-ab9b-fdca7ee2d9af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:29.923211 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f2bc4fca-6f48-4308-ab9b-fdca7ee2d9af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:29.925015 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f2bc4fca-6f48-4308-ab9b-fdca7ee2d9af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.206317 Aug 21 01:27:29.925575 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 145/287] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:29 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:27:29.986812 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3fff0f09-a3ba-4ee3-9782-32a736e91fb1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:29.986812 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3fff0f09-a3ba-4ee3-9782-32a736e91fb1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:29.990789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fff0f09-a3ba-4ee3-9782-32a736e91fb1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:29.991314 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fff0f09-a3ba-4ee3-9782-32a736e91fb1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:29.991762 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fff0f09-a3ba-4ee3-9782-32a736e91fb1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:29.992522 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fff0f09-a3ba-4ee3-9782-32a736e91fb1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:29.993624 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fff0f09-a3ba-4ee3-9782-32a736e91fb1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:29.993624 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fff0f09-a3ba-4ee3-9782-32a736e91fb1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:30.210576 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3fff0f09-a3ba-4ee3-9782-32a736e91fb1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:30.225046 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3fff0f09-a3ba-4ee3-9782-32a736e91fb1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.254974 Aug 21 01:27:30.228634 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 143/288] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:29 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:30.607540 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-929a03f9-7e72-4c9a-b86a-d0c5f35684e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:30.609429 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-929a03f9-7e72-4c9a-b86a-d0c5f35684e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:30.619218 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-929a03f9-7e72-4c9a-b86a-d0c5f35684e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:30.619672 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-929a03f9-7e72-4c9a-b86a-d0c5f35684e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:30.620797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-929a03f9-7e72-4c9a-b86a-d0c5f35684e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:30.621405 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-929a03f9-7e72-4c9a-b86a-d0c5f35684e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:30.622094 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-929a03f9-7e72-4c9a-b86a-d0c5f35684e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:30.622749 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-929a03f9-7e72-4c9a-b86a-d0c5f35684e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:30.691351 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-929a03f9-7e72-4c9a-b86a-d0c5f35684e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:30.697255 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-929a03f9-7e72-4c9a-b86a-d0c5f35684e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1296 microversion: 2.1 time: 0.094139 Aug 21 01:27:30.698081 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 146/289] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:30 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1296 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:30.951178 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-33764006-00b3-4934-9c64-38400c09ea98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:30.951788 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-33764006-00b3-4934-9c64-38400c09ea98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:30.959996 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-33764006-00b3-4934-9c64-38400c09ea98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:30.960509 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-33764006-00b3-4934-9c64-38400c09ea98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:30.965431 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-33764006-00b3-4934-9c64-38400c09ea98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:30.966252 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-33764006-00b3-4934-9c64-38400c09ea98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:30.966735 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-33764006-00b3-4934-9c64-38400c09ea98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:30.967198 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-33764006-00b3-4934-9c64-38400c09ea98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:31.159002 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-33764006-00b3-4934-9c64-38400c09ea98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:31.167981 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-33764006-00b3-4934-9c64-38400c09ea98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:31.169564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-33764006-00b3-4934-9c64-38400c09ea98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:31.171177 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-33764006-00b3-4934-9c64-38400c09ea98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.224018 Aug 21 01:27:31.171733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 144/290] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:30 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:31.262950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-60be7ce1-6acb-456d-917a-9fd2be7fcbf6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:31.263730 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-60be7ce1-6acb-456d-917a-9fd2be7fcbf6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:31.294573 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-60be7ce1-6acb-456d-917a-9fd2be7fcbf6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:31.295145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-60be7ce1-6acb-456d-917a-9fd2be7fcbf6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:31.313343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-60be7ce1-6acb-456d-917a-9fd2be7fcbf6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:31.313343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-60be7ce1-6acb-456d-917a-9fd2be7fcbf6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:31.313343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-60be7ce1-6acb-456d-917a-9fd2be7fcbf6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:31.313343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-60be7ce1-6acb-456d-917a-9fd2be7fcbf6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:31.418216 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-931746820", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452"}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:31.492255 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] validate_networks() for None {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:27:31.714570 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-60be7ce1-6acb-456d-917a-9fd2be7fcbf6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:31.729315 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-60be7ce1-6acb-456d-917a-9fd2be7fcbf6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.478305 Aug 21 01:27:31.744053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 147/291] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:31 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:31.744053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-59925bae-69ba-4a6f-a642-496de4a5a1b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:31.744053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-59925bae-69ba-4a6f-a642-496de4a5a1b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:31.746795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-59925bae-69ba-4a6f-a642-496de4a5a1b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:31.747286 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-59925bae-69ba-4a6f-a642-496de4a5a1b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:31.747752 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-59925bae-69ba-4a6f-a642-496de4a5a1b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:31.748555 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-59925bae-69ba-4a6f-a642-496de4a5a1b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:31.749014 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-59925bae-69ba-4a6f-a642-496de4a5a1b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:31.749594 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-59925bae-69ba-4a6f-a642-496de4a5a1b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:31.912078 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-59925bae-69ba-4a6f-a642-496de4a5a1b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:31.941196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-59925bae-69ba-4a6f-a642-496de4a5a1b9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1302 microversion: 2.1 time: 0.204673 Aug 21 01:27:31.941196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 148/292] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:31 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1302 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:32.197565 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-616c1846-13d8-4465-b117-e4694bd131b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:32.198390 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-616c1846-13d8-4465-b117-e4694bd131b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:32.212041 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-616c1846-13d8-4465-b117-e4694bd131b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:32.213182 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-616c1846-13d8-4465-b117-e4694bd131b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:32.216375 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-616c1846-13d8-4465-b117-e4694bd131b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:32.220406 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-616c1846-13d8-4465-b117-e4694bd131b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:32.221220 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-616c1846-13d8-4465-b117-e4694bd131b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:32.222723 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-616c1846-13d8-4465-b117-e4694bd131b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:32.423841 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:27:32.424472 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:27:32.424773 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:27:32.425374 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:27:32.425955 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:27:32.428297 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=63,nova_api:INSERT=3 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:32.447198 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:32.447764 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:32.448239 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:32.448978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:32.449440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:32.449862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:32.468377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Getting quotas for project e8775cdf1c9c40e8aac986896c4e0b01. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:27:32.473880 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Getting quotas for user 78a9f62090244baa883ca47efed0ae20 and project e8775cdf1c9c40e8aac986896c4e0b01. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:27:32.489495 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:27:32.491980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:27:32.637094 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-616c1846-13d8-4465-b117-e4694bd131b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:32.647519 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-616c1846-13d8-4465-b117-e4694bd131b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:32.649282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-616c1846-13d8-4465-b117-e4694bd131b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:32.650762 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-616c1846-13d8-4465-b117-e4694bd131b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.458584 Aug 21 01:27:32.651480 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 149/293] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:32 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:32.750985 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-08086ca4-bea1-4d75-aec5-ccbc12e0455b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:32.752225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-08086ca4-bea1-4d75-aec5-ccbc12e0455b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:32.761137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-08086ca4-bea1-4d75-aec5-ccbc12e0455b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:32.761636 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-08086ca4-bea1-4d75-aec5-ccbc12e0455b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:32.762077 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-08086ca4-bea1-4d75-aec5-ccbc12e0455b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:32.762782 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-08086ca4-bea1-4d75-aec5-ccbc12e0455b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:32.763231 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-08086ca4-bea1-4d75-aec5-ccbc12e0455b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:32.765858 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-08086ca4-bea1-4d75-aec5-ccbc12e0455b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:32.956812 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-08086ca4-bea1-4d75-aec5-ccbc12e0455b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:32.969771 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-08086ca4-bea1-4d75-aec5-ccbc12e0455b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.222644 Aug 21 01:27:32.976546 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-20e33bfd-120a-4c5a-97e1-03492d7725ba tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.559554 Aug 21 01:27:32.976546 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 145/294] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:27:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 1562 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:27:32.980622 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 150/295] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:32 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:27:32.988056 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e4985e9e-08b1-4258-94ce-7cbf579f3fda tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:32.988896 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e4985e9e-08b1-4258-94ce-7cbf579f3fda tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:32.993269 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c3f19c9a-a84e-43e1-b58e-fe58474ae1f5 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:32.994510 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c3f19c9a-a84e-43e1-b58e-fe58474ae1f5 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:32.997313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e4985e9e-08b1-4258-94ce-7cbf579f3fda tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:32.998142 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e4985e9e-08b1-4258-94ce-7cbf579f3fda tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:32.998583 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e4985e9e-08b1-4258-94ce-7cbf579f3fda tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:32.999296 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e4985e9e-08b1-4258-94ce-7cbf579f3fda tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:32.999750 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e4985e9e-08b1-4258-94ce-7cbf579f3fda tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:33.000431 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e4985e9e-08b1-4258-94ce-7cbf579f3fda tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:33.098418 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c3f19c9a-a84e-43e1-b58e-fe58474ae1f5 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:33.106658 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c3f19c9a-a84e-43e1-b58e-fe58474ae1f5 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1214 microversion: 2.1 time: 0.123254 Aug 21 01:27:33.110229 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 151/296] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:32 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1214 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:33.459567 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e4985e9e-08b1-4258-94ce-7cbf579f3fda tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:33.470056 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e4985e9e-08b1-4258-94ce-7cbf579f3fda tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.488098 Aug 21 01:27:33.477409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 146/297] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:32 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:33.674181 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-60daf7e9-48e3-4c0d-9149-e5b187bc7d98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:33.675539 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-60daf7e9-48e3-4c0d-9149-e5b187bc7d98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:33.699295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-60daf7e9-48e3-4c0d-9149-e5b187bc7d98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:33.707345 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-60daf7e9-48e3-4c0d-9149-e5b187bc7d98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:33.708119 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-60daf7e9-48e3-4c0d-9149-e5b187bc7d98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:33.709055 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-60daf7e9-48e3-4c0d-9149-e5b187bc7d98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:33.710664 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-60daf7e9-48e3-4c0d-9149-e5b187bc7d98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:33.715586 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-60daf7e9-48e3-4c0d-9149-e5b187bc7d98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:34.012937 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-15486a74-e8bd-4650-a848-574ded433c07 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:34.016622 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-15486a74-e8bd-4650-a848-574ded433c07 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:34.032149 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-15486a74-e8bd-4650-a848-574ded433c07 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:34.033520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-15486a74-e8bd-4650-a848-574ded433c07 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:34.034501 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-15486a74-e8bd-4650-a848-574ded433c07 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:34.040310 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-15486a74-e8bd-4650-a848-574ded433c07 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:34.041858 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-15486a74-e8bd-4650-a848-574ded433c07 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:34.042467 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-15486a74-e8bd-4650-a848-574ded433c07 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:34.429089 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-60daf7e9-48e3-4c0d-9149-e5b187bc7d98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:34.440176 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-60daf7e9-48e3-4c0d-9149-e5b187bc7d98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:34.442680 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-60daf7e9-48e3-4c0d-9149-e5b187bc7d98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:34.444005 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-60daf7e9-48e3-4c0d-9149-e5b187bc7d98 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.774117 Aug 21 01:27:34.457167 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 152/298] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:33 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 788 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:27:34.486780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b2fce34c-6bf8-4473-9b54-3b5ae57bf48d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:34.490029 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b2fce34c-6bf8-4473-9b54-3b5ae57bf48d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:34.516219 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b2fce34c-6bf8-4473-9b54-3b5ae57bf48d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:34.525730 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b2fce34c-6bf8-4473-9b54-3b5ae57bf48d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:34.526647 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b2fce34c-6bf8-4473-9b54-3b5ae57bf48d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:34.528030 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b2fce34c-6bf8-4473-9b54-3b5ae57bf48d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:34.529033 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b2fce34c-6bf8-4473-9b54-3b5ae57bf48d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:34.530172 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b2fce34c-6bf8-4473-9b54-3b5ae57bf48d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:34.542529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-15486a74-e8bd-4650-a848-574ded433c07 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:34.559137 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-15486a74-e8bd-4650-a848-574ded433c07 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.558823 Aug 21 01:27:34.588181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 147/299] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:33 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:34.593521 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8a71a6e5-c458-45e4-babf-86413348ac8a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:34.594568 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8a71a6e5-c458-45e4-babf-86413348ac8a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:34.608094 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a71a6e5-c458-45e4-babf-86413348ac8a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:34.613017 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a71a6e5-c458-45e4-babf-86413348ac8a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:34.613485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a71a6e5-c458-45e4-babf-86413348ac8a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:34.614237 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a71a6e5-c458-45e4-babf-86413348ac8a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:34.614705 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a71a6e5-c458-45e4-babf-86413348ac8a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:34.615138 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a71a6e5-c458-45e4-babf-86413348ac8a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:34.617047 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=100 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:34.645207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:34.731532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b2fce34c-6bf8-4473-9b54-3b5ae57bf48d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:34.741567 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b2fce34c-6bf8-4473-9b54-3b5ae57bf48d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1218 microversion: 2.1 time: 0.279853 Aug 21 01:27:34.743014 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 153/300] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:34 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1218 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:34.982642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8a71a6e5-c458-45e4-babf-86413348ac8a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:34.993191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8a71a6e5-c458-45e4-babf-86413348ac8a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.408340 Aug 21 01:27:35.007758 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 148/301] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:34 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:35.188215 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-25eec00b-7746-4ba4-84c8-febcd5183528 req-69c32467-9faf-489e-aa08-5a4fea776b8e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eddb6b3f-8ca0-4f71-b371-19c6e65dd095", "tag": "1e55ecc3-f194-4529-b75c-be5293dba5c9"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:35.211489 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-25eec00b-7746-4ba4-84c8-febcd5183528 req-69c32467-9faf-489e-aa08-5a4fea776b8e service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:35.212052 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-25eec00b-7746-4ba4-84c8-febcd5183528 req-69c32467-9faf-489e-aa08-5a4fea776b8e service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:35.214343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-25eec00b-7746-4ba4-84c8-febcd5183528 req-69c32467-9faf-489e-aa08-5a4fea776b8e service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:35.229506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-25eec00b-7746-4ba4-84c8-febcd5183528 req-69c32467-9faf-489e-aa08-5a4fea776b8e service nova] Creating event network-changed:1e55ecc3-f194-4529-b75c-be5293dba5c9 for instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 on hv-cinder-maste Aug 21 01:27:35.238430 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-25eec00b-7746-4ba4-84c8-febcd5183528 req-69c32467-9faf-489e-aa08-5a4fea776b8e service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053623 Aug 21 01:27:35.240026 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 154/302] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:27:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:27:35.476894 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-891720fd-9602-49e1-881d-bc1564350a1d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:35.478568 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-891720fd-9602-49e1-881d-bc1564350a1d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:35.501323 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-891720fd-9602-49e1-881d-bc1564350a1d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:35.501769 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-891720fd-9602-49e1-881d-bc1564350a1d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:35.503133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-891720fd-9602-49e1-881d-bc1564350a1d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:35.505662 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-891720fd-9602-49e1-881d-bc1564350a1d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:35.506286 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-891720fd-9602-49e1-881d-bc1564350a1d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:35.506859 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-891720fd-9602-49e1-881d-bc1564350a1d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:35.609777 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-22f95f30-3a6f-44d4-bc06-e914b93ee638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:35.610827 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-22f95f30-3a6f-44d4-bc06-e914b93ee638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:35.618876 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22f95f30-3a6f-44d4-bc06-e914b93ee638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:35.619665 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22f95f30-3a6f-44d4-bc06-e914b93ee638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:35.620356 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22f95f30-3a6f-44d4-bc06-e914b93ee638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:35.621519 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22f95f30-3a6f-44d4-bc06-e914b93ee638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:35.622178 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22f95f30-3a6f-44d4-bc06-e914b93ee638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:35.622785 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22f95f30-3a6f-44d4-bc06-e914b93ee638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:35.934137 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-891720fd-9602-49e1-881d-bc1564350a1d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:35.952338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-891720fd-9602-49e1-881d-bc1564350a1d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:35.969508 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-891720fd-9602-49e1-881d-bc1564350a1d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:35.970977 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-891720fd-9602-49e1-881d-bc1564350a1d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1743 microversion: 2.20 time: 0.505435 Aug 21 01:27:35.980515 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 149/303] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:35 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1743 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:35.988632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fa84cb23-8e09-46c2-946a-7f8089ae6839 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:35.989144 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fa84cb23-8e09-46c2-946a-7f8089ae6839 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:36.000798 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fa84cb23-8e09-46c2-946a-7f8089ae6839 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:36.001700 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fa84cb23-8e09-46c2-946a-7f8089ae6839 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:36.002425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fa84cb23-8e09-46c2-946a-7f8089ae6839 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:36.004350 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fa84cb23-8e09-46c2-946a-7f8089ae6839 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:36.005253 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fa84cb23-8e09-46c2-946a-7f8089ae6839 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:36.006091 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fa84cb23-8e09-46c2-946a-7f8089ae6839 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:36.051686 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-22f95f30-3a6f-44d4-bc06-e914b93ee638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:36.084641 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-22f95f30-3a6f-44d4-bc06-e914b93ee638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.479105 Aug 21 01:27:36.085970 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 155/304] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:35 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:36.091898 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-249c63fc-b005-4f28-99d1-151fb13e6d7f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:36.092967 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-249c63fc-b005-4f28-99d1-151fb13e6d7f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:36.106135 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-249c63fc-b005-4f28-99d1-151fb13e6d7f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:36.111663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-249c63fc-b005-4f28-99d1-151fb13e6d7f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:36.111663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-249c63fc-b005-4f28-99d1-151fb13e6d7f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:36.111663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-249c63fc-b005-4f28-99d1-151fb13e6d7f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:36.111663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-249c63fc-b005-4f28-99d1-151fb13e6d7f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:36.112443 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-249c63fc-b005-4f28-99d1-151fb13e6d7f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:36.203496 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fa84cb23-8e09-46c2-946a-7f8089ae6839 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:36.215029 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fa84cb23-8e09-46c2-946a-7f8089ae6839 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1266 microversion: 2.1 time: 0.232400 Aug 21 01:27:36.216636 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 150/305] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:35 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1266 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:36.406989 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-249c63fc-b005-4f28-99d1-151fb13e6d7f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:36.416618 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-249c63fc-b005-4f28-99d1-151fb13e6d7f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.328847 Aug 21 01:27:36.422162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 156/306] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:36 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:37.000182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6568391d-2df1-4f2f-bdaa-89989123ea3c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:37.001059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6568391d-2df1-4f2f-bdaa-89989123ea3c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:37.009867 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6568391d-2df1-4f2f-bdaa-89989123ea3c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:37.010580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6568391d-2df1-4f2f-bdaa-89989123ea3c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:37.011168 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6568391d-2df1-4f2f-bdaa-89989123ea3c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:37.012043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6568391d-2df1-4f2f-bdaa-89989123ea3c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:37.012682 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6568391d-2df1-4f2f-bdaa-89989123ea3c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:37.013268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6568391d-2df1-4f2f-bdaa-89989123ea3c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:37.108411 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4fee9afd-c570-40e0-972e-76d02a1244db tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:37.111203 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4fee9afd-c570-40e0-972e-76d02a1244db tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:37.123561 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4fee9afd-c570-40e0-972e-76d02a1244db tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:37.126852 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4fee9afd-c570-40e0-972e-76d02a1244db tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:37.127607 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4fee9afd-c570-40e0-972e-76d02a1244db tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:37.128773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4fee9afd-c570-40e0-972e-76d02a1244db tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:37.129268 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4fee9afd-c570-40e0-972e-76d02a1244db tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:37.129667 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4fee9afd-c570-40e0-972e-76d02a1244db tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:37.279236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6568391d-2df1-4f2f-bdaa-89989123ea3c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:37.291511 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6568391d-2df1-4f2f-bdaa-89989123ea3c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:37.295084 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6568391d-2df1-4f2f-bdaa-89989123ea3c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:37.297795 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6568391d-2df1-4f2f-bdaa-89989123ea3c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1743 microversion: 2.20 time: 0.301672 Aug 21 01:27:37.299006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 151/307] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:36 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1743 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:37.306239 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-99de07ec-edcf-4ee1-beae-f313f06d4429 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:37.306860 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-99de07ec-edcf-4ee1-beae-f313f06d4429 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:37.315004 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-99de07ec-edcf-4ee1-beae-f313f06d4429 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:37.315590 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-99de07ec-edcf-4ee1-beae-f313f06d4429 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:37.316140 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-99de07ec-edcf-4ee1-beae-f313f06d4429 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:37.316872 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-99de07ec-edcf-4ee1-beae-f313f06d4429 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:37.317521 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-99de07ec-edcf-4ee1-beae-f313f06d4429 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:37.317950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-99de07ec-edcf-4ee1-beae-f313f06d4429 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:37.396070 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-99de07ec-edcf-4ee1-beae-f313f06d4429 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:37.400543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4fee9afd-c570-40e0-972e-76d02a1244db tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:37.404927 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-99de07ec-edcf-4ee1-beae-f313f06d4429 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1284 microversion: 2.1 time: 0.104530 Aug 21 01:27:37.405935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 152/308] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:37 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1284 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:37.415798 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4fee9afd-c570-40e0-972e-76d02a1244db tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.312404 Aug 21 01:27:37.416830 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 157/309] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:37 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:37.445132 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8d18a717-5060-4abb-9d21-0bfdd82f34b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:37.445828 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8d18a717-5060-4abb-9d21-0bfdd82f34b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:37.455088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8d18a717-5060-4abb-9d21-0bfdd82f34b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:37.455611 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8d18a717-5060-4abb-9d21-0bfdd82f34b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:37.456059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8d18a717-5060-4abb-9d21-0bfdd82f34b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:37.457316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8d18a717-5060-4abb-9d21-0bfdd82f34b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:37.457787 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8d18a717-5060-4abb-9d21-0bfdd82f34b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:37.458225 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8d18a717-5060-4abb-9d21-0bfdd82f34b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:37.608915 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8d18a717-5060-4abb-9d21-0bfdd82f34b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:37.616999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8d18a717-5060-4abb-9d21-0bfdd82f34b2 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.175736 Aug 21 01:27:37.617882 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 153/310] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:37 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:38.326058 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8b91f933-778c-4525-a3b7-37d5087ee6dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:38.330734 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8b91f933-778c-4525-a3b7-37d5087ee6dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:38.337733 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8b91f933-778c-4525-a3b7-37d5087ee6dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:38.338230 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8b91f933-778c-4525-a3b7-37d5087ee6dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:38.338900 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8b91f933-778c-4525-a3b7-37d5087ee6dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:38.339878 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8b91f933-778c-4525-a3b7-37d5087ee6dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:38.340369 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8b91f933-778c-4525-a3b7-37d5087ee6dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:38.340804 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8b91f933-778c-4525-a3b7-37d5087ee6dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:38.431090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-495f851e-6748-4bbc-99e2-4f59f9e84adb tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:38.437378 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-495f851e-6748-4bbc-99e2-4f59f9e84adb tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:38.447316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-495f851e-6748-4bbc-99e2-4f59f9e84adb tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:38.448093 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-495f851e-6748-4bbc-99e2-4f59f9e84adb tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:38.448602 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-495f851e-6748-4bbc-99e2-4f59f9e84adb tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:38.449553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-495f851e-6748-4bbc-99e2-4f59f9e84adb tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:38.450346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-495f851e-6748-4bbc-99e2-4f59f9e84adb tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:38.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-495f851e-6748-4bbc-99e2-4f59f9e84adb tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:38.516843 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8b91f933-778c-4525-a3b7-37d5087ee6dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:38.531373 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8b91f933-778c-4525-a3b7-37d5087ee6dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:38.538928 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8b91f933-778c-4525-a3b7-37d5087ee6dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:38.540938 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8b91f933-778c-4525-a3b7-37d5087ee6dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1743 microversion: 2.20 time: 0.219125 Aug 21 01:27:38.544723 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 158/311] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:38 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1743 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:38.559334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-92f96fcd-f71d-4b92-91b6-ea99d85f260f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:38.559995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-92f96fcd-f71d-4b92-91b6-ea99d85f260f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:38.567272 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-92f96fcd-f71d-4b92-91b6-ea99d85f260f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:38.568171 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-92f96fcd-f71d-4b92-91b6-ea99d85f260f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:38.568899 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-92f96fcd-f71d-4b92-91b6-ea99d85f260f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:38.569887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-92f96fcd-f71d-4b92-91b6-ea99d85f260f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:38.571630 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-92f96fcd-f71d-4b92-91b6-ea99d85f260f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:38.572311 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-92f96fcd-f71d-4b92-91b6-ea99d85f260f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:38.586099 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-495f851e-6748-4bbc-99e2-4f59f9e84adb tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:38.597296 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-495f851e-6748-4bbc-99e2-4f59f9e84adb tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1284 microversion: 2.1 time: 0.169859 Aug 21 01:27:38.600378 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 154/312] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:38 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1284 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:27:38.638925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-5638623c-eb9a-4fcd-a51c-7d36035696ab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:38.639556 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-5638623c-eb9a-4fcd-a51c-7d36035696ab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:38.655342 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5638623c-eb9a-4fcd-a51c-7d36035696ab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:38.655848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5638623c-eb9a-4fcd-a51c-7d36035696ab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:38.656333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5638623c-eb9a-4fcd-a51c-7d36035696ab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:38.657071 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5638623c-eb9a-4fcd-a51c-7d36035696ab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:38.657536 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5638623c-eb9a-4fcd-a51c-7d36035696ab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:38.657964 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5638623c-eb9a-4fcd-a51c-7d36035696ab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:38.832094 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-92f96fcd-f71d-4b92-91b6-ea99d85f260f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:38.841550 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-92f96fcd-f71d-4b92-91b6-ea99d85f260f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.295365 Aug 21 01:27:38.842276 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 159/313] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:38 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:38.908304 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5638623c-eb9a-4fcd-a51c-7d36035696ab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:38.920542 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5638623c-eb9a-4fcd-a51c-7d36035696ab tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.285392 Aug 21 01:27:38.920924 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 155/314] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:38 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:27:39.569672 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f4f0af43-31a2-45c9-a8d3-175632cc22f9 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:39.570293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f4f0af43-31a2-45c9-a8d3-175632cc22f9 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:39.590049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f4f0af43-31a2-45c9-a8d3-175632cc22f9 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:39.590049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f4f0af43-31a2-45c9-a8d3-175632cc22f9 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:39.590049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f4f0af43-31a2-45c9-a8d3-175632cc22f9 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:39.591580 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f4f0af43-31a2-45c9-a8d3-175632cc22f9 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:39.592319 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f4f0af43-31a2-45c9-a8d3-175632cc22f9 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:39.593495 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f4f0af43-31a2-45c9-a8d3-175632cc22f9 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:39.630501 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-da57502e-2425-48f7-a26a-55ff30d311c0 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:39.631241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-da57502e-2425-48f7-a26a-55ff30d311c0 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:39.650258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-da57502e-2425-48f7-a26a-55ff30d311c0 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:39.651624 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-da57502e-2425-48f7-a26a-55ff30d311c0 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:39.653289 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-da57502e-2425-48f7-a26a-55ff30d311c0 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:39.657990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-da57502e-2425-48f7-a26a-55ff30d311c0 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:39.657990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-da57502e-2425-48f7-a26a-55ff30d311c0 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:39.657990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-da57502e-2425-48f7-a26a-55ff30d311c0 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:39.857533 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-da57502e-2425-48f7-a26a-55ff30d311c0 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:39.873677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-da57502e-2425-48f7-a26a-55ff30d311c0 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1284 microversion: 2.1 time: 0.250319 Aug 21 01:27:39.874344 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 156/315] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:39 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1284 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:27:39.911349 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d8f66cc0-dd74-41f9-ae1c-d16050bc1a2d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:39.911806 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d8f66cc0-dd74-41f9-ae1c-d16050bc1a2d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:39.950143 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d8f66cc0-dd74-41f9-ae1c-d16050bc1a2d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:39.950688 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d8f66cc0-dd74-41f9-ae1c-d16050bc1a2d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:39.959257 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d8f66cc0-dd74-41f9-ae1c-d16050bc1a2d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:39.963016 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d8f66cc0-dd74-41f9-ae1c-d16050bc1a2d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:39.963709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d8f66cc0-dd74-41f9-ae1c-d16050bc1a2d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:39.964413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d8f66cc0-dd74-41f9-ae1c-d16050bc1a2d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:40.325941 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f4f0af43-31a2-45c9-a8d3-175632cc22f9 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:40.341248 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f4f0af43-31a2-45c9-a8d3-175632cc22f9 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:40.343184 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f4f0af43-31a2-45c9-a8d3-175632cc22f9 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:40.344679 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f4f0af43-31a2-45c9-a8d3-175632cc22f9 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1743 microversion: 2.20 time: 0.780709 Aug 21 01:27:40.366341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 160/316] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:39 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1743 bytes in 802 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:40.373168 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-168e1d33-03e3-42fc-abae-378d28f995e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:40.373949 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-168e1d33-03e3-42fc-abae-378d28f995e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:40.383716 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-168e1d33-03e3-42fc-abae-378d28f995e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:40.385294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-168e1d33-03e3-42fc-abae-378d28f995e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:40.386257 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-168e1d33-03e3-42fc-abae-378d28f995e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:40.387111 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-168e1d33-03e3-42fc-abae-378d28f995e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:40.387775 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-168e1d33-03e3-42fc-abae-378d28f995e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:40.389641 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-168e1d33-03e3-42fc-abae-378d28f995e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:40.457216 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d8f66cc0-dd74-41f9-ae1c-d16050bc1a2d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:40.470837 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d8f66cc0-dd74-41f9-ae1c-d16050bc1a2d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.595337 Aug 21 01:27:40.474252 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 157/317] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:39 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:40.644461 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-168e1d33-03e3-42fc-abae-378d28f995e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:40.657881 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-168e1d33-03e3-42fc-abae-378d28f995e4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.287799 Aug 21 01:27:40.659014 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 161/318] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:40 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:40.907878 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7e714f18-0f7d-4ef7-a352-7b37bdb27028 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:40.908590 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7e714f18-0f7d-4ef7-a352-7b37bdb27028 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:40.919693 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7e714f18-0f7d-4ef7-a352-7b37bdb27028 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:40.922627 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7e714f18-0f7d-4ef7-a352-7b37bdb27028 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:40.923203 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7e714f18-0f7d-4ef7-a352-7b37bdb27028 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:40.924228 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7e714f18-0f7d-4ef7-a352-7b37bdb27028 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:40.924707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7e714f18-0f7d-4ef7-a352-7b37bdb27028 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:40.929562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7e714f18-0f7d-4ef7-a352-7b37bdb27028 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:41.073854 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7e714f18-0f7d-4ef7-a352-7b37bdb27028 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:41.088912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7e714f18-0f7d-4ef7-a352-7b37bdb27028 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1272 microversion: 2.1 time: 0.192915 Aug 21 01:27:41.092281 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 158/319] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:40 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1272 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:41.377509 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-12af5d33-6f30-4ab2-85b8-0999386d204b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:41.378536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-12af5d33-6f30-4ab2-85b8-0999386d204b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:41.386813 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-12af5d33-6f30-4ab2-85b8-0999386d204b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:41.387624 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-12af5d33-6f30-4ab2-85b8-0999386d204b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:41.388280 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-12af5d33-6f30-4ab2-85b8-0999386d204b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:41.389191 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-12af5d33-6f30-4ab2-85b8-0999386d204b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:41.389875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-12af5d33-6f30-4ab2-85b8-0999386d204b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:41.390479 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-12af5d33-6f30-4ab2-85b8-0999386d204b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:41.495763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-33a889f9-07f8-4669-b00f-5dacb23b95c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:41.496407 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-33a889f9-07f8-4669-b00f-5dacb23b95c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:41.502707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-33a889f9-07f8-4669-b00f-5dacb23b95c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:41.503204 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-33a889f9-07f8-4669-b00f-5dacb23b95c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:41.503643 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-33a889f9-07f8-4669-b00f-5dacb23b95c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:41.504945 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-33a889f9-07f8-4669-b00f-5dacb23b95c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:41.505417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-33a889f9-07f8-4669-b00f-5dacb23b95c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:41.505845 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-33a889f9-07f8-4669-b00f-5dacb23b95c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:41.782924 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-12af5d33-6f30-4ab2-85b8-0999386d204b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:41.797347 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-12af5d33-6f30-4ab2-85b8-0999386d204b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:41.806482 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-12af5d33-6f30-4ab2-85b8-0999386d204b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:41.810208 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-33a889f9-07f8-4669-b00f-5dacb23b95c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:41.811386 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-12af5d33-6f30-4ab2-85b8-0999386d204b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.441290 Aug 21 01:27:41.816470 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 162/320] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:41 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:41.830999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9996e98a-bc5f-4c70-932a-1ee0b1b84c43 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:41.831542 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9996e98a-bc5f-4c70-932a-1ee0b1b84c43 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:41.837866 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-33a889f9-07f8-4669-b00f-5dacb23b95c3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.345784 Aug 21 01:27:41.840401 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 159/321] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:41 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:27:41.866803 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9996e98a-bc5f-4c70-932a-1ee0b1b84c43 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:41.867576 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9996e98a-bc5f-4c70-932a-1ee0b1b84c43 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:41.868405 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9996e98a-bc5f-4c70-932a-1ee0b1b84c43 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:41.869397 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9996e98a-bc5f-4c70-932a-1ee0b1b84c43 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:41.870071 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9996e98a-bc5f-4c70-932a-1ee0b1b84c43 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:41.870748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9996e98a-bc5f-4c70-932a-1ee0b1b84c43 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:42.113693 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a1db7d8c-f561-4762-8d74-dccec39dad2d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:42.114436 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a1db7d8c-f561-4762-8d74-dccec39dad2d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:42.133250 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a1db7d8c-f561-4762-8d74-dccec39dad2d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:42.134002 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a1db7d8c-f561-4762-8d74-dccec39dad2d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:42.134604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a1db7d8c-f561-4762-8d74-dccec39dad2d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:42.135505 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a1db7d8c-f561-4762-8d74-dccec39dad2d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:42.136170 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a1db7d8c-f561-4762-8d74-dccec39dad2d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:42.136756 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a1db7d8c-f561-4762-8d74-dccec39dad2d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:42.340562 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9996e98a-bc5f-4c70-932a-1ee0b1b84c43 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:42.365394 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9996e98a-bc5f-4c70-932a-1ee0b1b84c43 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.533326 Aug 21 01:27:42.369046 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 163/322] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:41 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:42.460718 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=4 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:42.473754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a1db7d8c-f561-4762-8d74-dccec39dad2d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:42.482471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=12 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:42.485690 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a1db7d8c-f561-4762-8d74-dccec39dad2d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1314 microversion: 2.1 time: 0.380283 Aug 21 01:27:42.501072 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 160/323] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:42 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1314 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:42.837423 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8c8f3b9c-bd39-4433-835f-d1e1b94aaa7e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:42.838598 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8c8f3b9c-bd39-4433-835f-d1e1b94aaa7e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:42.849592 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c8f3b9c-bd39-4433-835f-d1e1b94aaa7e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:42.850448 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c8f3b9c-bd39-4433-835f-d1e1b94aaa7e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:42.851085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c8f3b9c-bd39-4433-835f-d1e1b94aaa7e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:42.852050 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c8f3b9c-bd39-4433-835f-d1e1b94aaa7e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:42.853273 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c8f3b9c-bd39-4433-835f-d1e1b94aaa7e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:42.853869 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c8f3b9c-bd39-4433-835f-d1e1b94aaa7e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:42.864937 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1c61779c-7f69-4e07-81c1-f330360413eb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:42.865625 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1c61779c-7f69-4e07-81c1-f330360413eb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:42.882632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c61779c-7f69-4e07-81c1-f330360413eb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:42.883135 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c61779c-7f69-4e07-81c1-f330360413eb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:42.887917 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c61779c-7f69-4e07-81c1-f330360413eb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:42.888816 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c61779c-7f69-4e07-81c1-f330360413eb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:42.889769 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c61779c-7f69-4e07-81c1-f330360413eb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:42.890968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c61779c-7f69-4e07-81c1-f330360413eb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:43.103219 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=8 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:43.543270 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8c8f3b9c-bd39-4433-835f-d1e1b94aaa7e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:43.557270 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8c8f3b9c-bd39-4433-835f-d1e1b94aaa7e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:43.562305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8c8f3b9c-bd39-4433-835f-d1e1b94aaa7e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:43.565896 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8c8f3b9c-bd39-4433-835f-d1e1b94aaa7e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.732214 Aug 21 01:27:43.601462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1c61779c-7f69-4e07-81c1-f330360413eb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:43.619834 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 164/324] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:42 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 786 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:43.629687 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-59220336-e547-45d4-bf13-f89cc3caada3 req-198a0c26-8eed-4762-b6ea-6bb11e3afeb4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1", "name": "network-vif-plugged", "status": "completed", "tag": "9a09b4f3-32e5-4362-82ef-ab73b634b1bf"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:43.650908 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-59220336-e547-45d4-bf13-f89cc3caada3 req-198a0c26-8eed-4762-b6ea-6bb11e3afeb4 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:43.651713 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-59220336-e547-45d4-bf13-f89cc3caada3 req-198a0c26-8eed-4762-b6ea-6bb11e3afeb4 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:43.653104 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-59220336-e547-45d4-bf13-f89cc3caada3 req-198a0c26-8eed-4762-b6ea-6bb11e3afeb4 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:43.667199 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-59220336-e547-45d4-bf13-f89cc3caada3 req-198a0c26-8eed-4762-b6ea-6bb11e3afeb4 service nova] Creating event network-vif-plugged:9a09b4f3-32e5-4362-82ef-ab73b634b1bf for instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 on hv-cinder-maste Aug 21 01:27:43.678591 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-59220336-e547-45d4-bf13-f89cc3caada3 req-198a0c26-8eed-4762-b6ea-6bb11e3afeb4 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054223 Aug 21 01:27:43.679560 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 165/325] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:27:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:27:43.690263 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e2f8eff3-4385-42a5-9bae-ec8b3f47502a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:43.690976 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e2f8eff3-4385-42a5-9bae-ec8b3f47502a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:43.696345 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1c61779c-7f69-4e07-81c1-f330360413eb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.805068 Aug 21 01:27:43.705345 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 161/326] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:42 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 839 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:43.710394 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e2f8eff3-4385-42a5-9bae-ec8b3f47502a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:43.710909 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e2f8eff3-4385-42a5-9bae-ec8b3f47502a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:43.711370 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e2f8eff3-4385-42a5-9bae-ec8b3f47502a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:43.712085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e2f8eff3-4385-42a5-9bae-ec8b3f47502a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:43.717064 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e2f8eff3-4385-42a5-9bae-ec8b3f47502a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:43.717833 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e2f8eff3-4385-42a5-9bae-ec8b3f47502a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:43.719150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-dbeeb328-9dd6-4077-84ea-1081d133b283 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:43.741819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-dbeeb328-9dd6-4077-84ea-1081d133b283 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:43.756305 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbeeb328-9dd6-4077-84ea-1081d133b283 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:43.760550 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbeeb328-9dd6-4077-84ea-1081d133b283 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:43.760550 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbeeb328-9dd6-4077-84ea-1081d133b283 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:43.765422 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbeeb328-9dd6-4077-84ea-1081d133b283 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:43.768294 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbeeb328-9dd6-4077-84ea-1081d133b283 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:43.773908 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbeeb328-9dd6-4077-84ea-1081d133b283 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:44.237765 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e2f8eff3-4385-42a5-9bae-ec8b3f47502a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:44.241226 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-dbeeb328-9dd6-4077-84ea-1081d133b283 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:44.251744 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-dbeeb328-9dd6-4077-84ea-1081d133b283 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1314 microversion: 2.1 time: 0.552489 Aug 21 01:27:44.266670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e2f8eff3-4385-42a5-9bae-ec8b3f47502a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.566114 Aug 21 01:27:44.266670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 166/327] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:43 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:44.266952 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 162/328] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:43 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1314 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:44.624137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-92520a5a-ab73-469a-be75-73413f0b7906 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:44.625250 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-92520a5a-ab73-469a-be75-73413f0b7906 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:44.650738 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-92520a5a-ab73-469a-be75-73413f0b7906 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:44.651569 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-92520a5a-ab73-469a-be75-73413f0b7906 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:44.652237 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-92520a5a-ab73-469a-be75-73413f0b7906 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:44.653152 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-92520a5a-ab73-469a-be75-73413f0b7906 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:44.653798 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-92520a5a-ab73-469a-be75-73413f0b7906 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:44.654403 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-92520a5a-ab73-469a-be75-73413f0b7906 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:44.749095 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a73d12be-7a13-4805-b35a-f0f4ff0e220d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:44.749808 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a73d12be-7a13-4805-b35a-f0f4ff0e220d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:44.762235 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a73d12be-7a13-4805-b35a-f0f4ff0e220d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:44.764514 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a73d12be-7a13-4805-b35a-f0f4ff0e220d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:44.765029 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a73d12be-7a13-4805-b35a-f0f4ff0e220d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:44.773049 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a73d12be-7a13-4805-b35a-f0f4ff0e220d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:44.773539 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a73d12be-7a13-4805-b35a-f0f4ff0e220d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:44.773980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a73d12be-7a13-4805-b35a-f0f4ff0e220d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:45.219194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-92520a5a-ab73-469a-be75-73413f0b7906 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:45.249535 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-92520a5a-ab73-469a-be75-73413f0b7906 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:45.253074 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-92520a5a-ab73-469a-be75-73413f0b7906 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:45.261412 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-92520a5a-ab73-469a-be75-73413f0b7906 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.644217 Aug 21 01:27:45.265056 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 167/329] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:44 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:45.288454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-28bae0b7-3c0d-4de6-951a-553f84700a17 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:45.293622 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-28bae0b7-3c0d-4de6-951a-553f84700a17 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:45.326586 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-28bae0b7-3c0d-4de6-951a-553f84700a17 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:45.327847 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-28bae0b7-3c0d-4de6-951a-553f84700a17 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:45.330181 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-28bae0b7-3c0d-4de6-951a-553f84700a17 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:45.332227 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-28bae0b7-3c0d-4de6-951a-553f84700a17 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:45.334346 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-28bae0b7-3c0d-4de6-951a-553f84700a17 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:45.335578 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-28bae0b7-3c0d-4de6-951a-553f84700a17 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:45.478702 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a73d12be-7a13-4805-b35a-f0f4ff0e220d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:45.499822 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a73d12be-7a13-4805-b35a-f0f4ff0e220d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.754736 Aug 21 01:27:45.500562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 163/330] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:44 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 756 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:27:45.505101 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9734468d-fabf-4cc1-88ae-ad92326d2ffe tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:45.505837 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9734468d-fabf-4cc1-88ae-ad92326d2ffe tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:45.523393 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9734468d-fabf-4cc1-88ae-ad92326d2ffe tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:45.524146 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9734468d-fabf-4cc1-88ae-ad92326d2ffe tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:45.524732 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9734468d-fabf-4cc1-88ae-ad92326d2ffe tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:45.525579 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9734468d-fabf-4cc1-88ae-ad92326d2ffe tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:45.534002 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9734468d-fabf-4cc1-88ae-ad92326d2ffe tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:45.534856 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9734468d-fabf-4cc1-88ae-ad92326d2ffe tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:45.862610 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-28bae0b7-3c0d-4de6-951a-553f84700a17 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:45.876248 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-28bae0b7-3c0d-4de6-951a-553f84700a17 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1314 microversion: 2.1 time: 0.597416 Aug 21 01:27:45.876248 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 168/331] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:45 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1314 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:27:45.897073 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-8493d133-e8e1-4a9d-91c8-8eee0925af35 req-cf03018d-66db-4d34-9468-b97b3327ab63 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "09d5a793-bbb6-4169-b790-835069573e2a", "tag": "1dad426a-d53f-4f11-9c48-ba0aa747d6c3"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:45.925279 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:45.927744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-8493d133-e8e1-4a9d-91c8-8eee0925af35 req-cf03018d-66db-4d34-9468-b97b3327ab63 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:45.928287 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-8493d133-e8e1-4a9d-91c8-8eee0925af35 req-cf03018d-66db-4d34-9468-b97b3327ab63 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:45.928909 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-8493d133-e8e1-4a9d-91c8-8eee0925af35 req-cf03018d-66db-4d34-9468-b97b3327ab63 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:45.959717 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-8493d133-e8e1-4a9d-91c8-8eee0925af35 req-cf03018d-66db-4d34-9468-b97b3327ab63 service nova] Creating event network-changed:1dad426a-d53f-4f11-9c48-ba0aa747d6c3 for instance 09d5a793-bbb6-4169-b790-835069573e2a on hv-cinder-maste Aug 21 01:27:45.982869 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-8493d133-e8e1-4a9d-91c8-8eee0925af35 req-cf03018d-66db-4d34-9468-b97b3327ab63 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089378 Aug 21 01:27:45.984139 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 169/332] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:27:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:27:46.113909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9734468d-fabf-4cc1-88ae-ad92326d2ffe tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:46.134427 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9734468d-fabf-4cc1-88ae-ad92326d2ffe tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.632702 Aug 21 01:27:46.152812 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 164/333] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:45 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:46.313122 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-dfb3ad0b-9930-4e31-9d60-3bb960dc2cce tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:46.321135 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-dfb3ad0b-9930-4e31-9d60-3bb960dc2cce tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:46.369320 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dfb3ad0b-9930-4e31-9d60-3bb960dc2cce tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:46.381334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dfb3ad0b-9930-4e31-9d60-3bb960dc2cce tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:46.382645 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dfb3ad0b-9930-4e31-9d60-3bb960dc2cce tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:46.384432 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dfb3ad0b-9930-4e31-9d60-3bb960dc2cce tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:46.385702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dfb3ad0b-9930-4e31-9d60-3bb960dc2cce tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:46.386742 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dfb3ad0b-9930-4e31-9d60-3bb960dc2cce tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:46.553050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-21c8a05b-85cb-44f3-8aa7-3d2798c0e3ad tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:46.558153 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-21c8a05b-85cb-44f3-8aa7-3d2798c0e3ad tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:46.601468 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-21c8a05b-85cb-44f3-8aa7-3d2798c0e3ad tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:46.609431 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-21c8a05b-85cb-44f3-8aa7-3d2798c0e3ad tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:46.612669 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-21c8a05b-85cb-44f3-8aa7-3d2798c0e3ad tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:46.615960 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-21c8a05b-85cb-44f3-8aa7-3d2798c0e3ad tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:46.618214 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-21c8a05b-85cb-44f3-8aa7-3d2798c0e3ad tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:46.619239 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-21c8a05b-85cb-44f3-8aa7-3d2798c0e3ad tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:46.640930 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=100 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:47.411016 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-dfb3ad0b-9930-4e31-9d60-3bb960dc2cce tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:47.420544 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-dfb3ad0b-9930-4e31-9d60-3bb960dc2cce tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:47.440930 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-dfb3ad0b-9930-4e31-9d60-3bb960dc2cce tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:47.442353 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-dfb3ad0b-9930-4e31-9d60-3bb960dc2cce tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 1.134889 Aug 21 01:27:47.459092 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 170/334] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:46 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 1137 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:47.463635 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3f45a7ff-0d67-4890-aafa-f45e7f3c0d20 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:47.467387 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3f45a7ff-0d67-4890-aafa-f45e7f3c0d20 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:47.483572 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3f45a7ff-0d67-4890-aafa-f45e7f3c0d20 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:47.484393 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3f45a7ff-0d67-4890-aafa-f45e7f3c0d20 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:47.486326 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3f45a7ff-0d67-4890-aafa-f45e7f3c0d20 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:47.489137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3f45a7ff-0d67-4890-aafa-f45e7f3c0d20 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:47.492926 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3f45a7ff-0d67-4890-aafa-f45e7f3c0d20 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:47.493651 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3f45a7ff-0d67-4890-aafa-f45e7f3c0d20 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:47.609977 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-21c8a05b-85cb-44f3-8aa7-3d2798c0e3ad tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:47.633394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-21c8a05b-85cb-44f3-8aa7-3d2798c0e3ad tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1702 microversion: 2.1 time: 1.093897 Aug 21 01:27:47.633986 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 165/335] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:46 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1702 bytes in 1098 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:47.666236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e174eb40-1cc9-4f69-9b3f-da8b5c72f245 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:47.667181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e174eb40-1cc9-4f69-9b3f-da8b5c72f245 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:47.681878 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e174eb40-1cc9-4f69-9b3f-da8b5c72f245 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:47.682379 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e174eb40-1cc9-4f69-9b3f-da8b5c72f245 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:47.682826 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e174eb40-1cc9-4f69-9b3f-da8b5c72f245 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:47.683542 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e174eb40-1cc9-4f69-9b3f-da8b5c72f245 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:47.684009 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e174eb40-1cc9-4f69-9b3f-da8b5c72f245 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:47.684468 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e174eb40-1cc9-4f69-9b3f-da8b5c72f245 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:48.020151 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3f45a7ff-0d67-4890-aafa-f45e7f3c0d20 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:48.065513 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3f45a7ff-0d67-4890-aafa-f45e7f3c0d20 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1314 microversion: 2.1 time: 0.620112 Aug 21 01:27:48.066790 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 171/336] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:47 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1314 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:48.100213 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-73caa948-1f1c-4436-a7f8-66f0ffd13e03 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:48.109585 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-73caa948-1f1c-4436-a7f8-66f0ffd13e03 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:48.110388 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-73caa948-1f1c-4436-a7f8-66f0ffd13e03 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:48.111025 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-73caa948-1f1c-4436-a7f8-66f0ffd13e03 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:48.154204 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e174eb40-1cc9-4f69-9b3f-da8b5c72f245 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:48.162488 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e174eb40-1cc9-4f69-9b3f-da8b5c72f245 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.522204 Aug 21 01:27:48.163701 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 166/337] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:47 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:48.185912 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=100 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:48.261365 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-73caa948-1f1c-4436-a7f8-66f0ffd13e03 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Going to try to start instance {{(pid=121277) start /opt/stack/nova/nova/compute/api.py:2772}} Aug 21 01:27:48.468789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-87118fc6-c64b-4bb5-a6e4-d3f3920c9cc1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:48.469565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-87118fc6-c64b-4bb5-a6e4-d3f3920c9cc1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:48.485980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-87118fc6-c64b-4bb5-a6e4-d3f3920c9cc1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:48.487359 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-87118fc6-c64b-4bb5-a6e4-d3f3920c9cc1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:48.488069 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-87118fc6-c64b-4bb5-a6e4-d3f3920c9cc1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:48.489221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-87118fc6-c64b-4bb5-a6e4-d3f3920c9cc1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:48.490584 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-87118fc6-c64b-4bb5-a6e4-d3f3920c9cc1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:48.491238 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-87118fc6-c64b-4bb5-a6e4-d3f3920c9cc1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:48.550252 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-73caa948-1f1c-4436-a7f8-66f0ffd13e03 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "POST /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/action" status: 202 len: 0 microversion: 2.1 time: 0.455571 Aug 21 01:27:48.550938 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 172/338] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:27:48 2022] POST /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/action => generated 0 bytes in 483 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:27:48.561542 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-fecbaaa7-93ee-4dde-83a1-93d8056fca52 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:48.562252 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-fecbaaa7-93ee-4dde-83a1-93d8056fca52 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:48.579658 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fecbaaa7-93ee-4dde-83a1-93d8056fca52 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:48.580461 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fecbaaa7-93ee-4dde-83a1-93d8056fca52 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:48.592895 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fecbaaa7-93ee-4dde-83a1-93d8056fca52 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:48.594056 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fecbaaa7-93ee-4dde-83a1-93d8056fca52 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:48.594689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fecbaaa7-93ee-4dde-83a1-93d8056fca52 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:48.595270 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fecbaaa7-93ee-4dde-83a1-93d8056fca52 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:48.917709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-87118fc6-c64b-4bb5-a6e4-d3f3920c9cc1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:48.929165 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-87118fc6-c64b-4bb5-a6e4-d3f3920c9cc1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:48.931048 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-87118fc6-c64b-4bb5-a6e4-d3f3920c9cc1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:48.932480 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-87118fc6-c64b-4bb5-a6e4-d3f3920c9cc1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.467814 Aug 21 01:27:48.933531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 167/339] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:48 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:27:49.077461 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-fecbaaa7-93ee-4dde-83a1-93d8056fca52 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:49.095754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-fecbaaa7-93ee-4dde-83a1-93d8056fca52 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1711 microversion: 2.1 time: 0.538278 Aug 21 01:27:49.101902 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-71a77e47-42eb-4008-850e-0866d6741e00 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:49.102940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-71a77e47-42eb-4008-850e-0866d6741e00 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:49.109444 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 173/340] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:48 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1711 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:27:49.121429 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-71a77e47-42eb-4008-850e-0866d6741e00 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:49.122659 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-71a77e47-42eb-4008-850e-0866d6741e00 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:49.124289 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-71a77e47-42eb-4008-850e-0866d6741e00 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:49.125390 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-71a77e47-42eb-4008-850e-0866d6741e00 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:49.126186 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-71a77e47-42eb-4008-850e-0866d6741e00 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:49.126785 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-71a77e47-42eb-4008-850e-0866d6741e00 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:49.187673 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8c6d1174-34bf-4404-a1db-dd9f4f7f7fcc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:49.188319 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8c6d1174-34bf-4404-a1db-dd9f4f7f7fcc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:49.204851 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c6d1174-34bf-4404-a1db-dd9f4f7f7fcc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:49.205349 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c6d1174-34bf-4404-a1db-dd9f4f7f7fcc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:49.205787 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c6d1174-34bf-4404-a1db-dd9f4f7f7fcc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:49.206498 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c6d1174-34bf-4404-a1db-dd9f4f7f7fcc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:49.206954 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c6d1174-34bf-4404-a1db-dd9f4f7f7fcc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:49.207373 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c6d1174-34bf-4404-a1db-dd9f4f7f7fcc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:49.735121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-71a77e47-42eb-4008-850e-0866d6741e00 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:49.753048 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8c6d1174-34bf-4404-a1db-dd9f4f7f7fcc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:49.780342 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-71a77e47-42eb-4008-850e-0866d6741e00 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1314 microversion: 2.1 time: 0.685041 Aug 21 01:27:49.789542 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8c6d1174-34bf-4404-a1db-dd9f4f7f7fcc tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.605631 Aug 21 01:27:49.798077 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 168/341] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:49 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1314 bytes in 704 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:49.803992 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-5f39b2ca-4f13-4346-b14c-8647612e4f1c req-887fca6b-3ee5-42a9-869b-725d134d883c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eddb6b3f-8ca0-4f71-b371-19c6e65dd095", "name": "network-vif-plugged", "status": "completed", "tag": "1e55ecc3-f194-4529-b75c-be5293dba5c9"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:49.809644 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 174/342] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:49 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:49.836868 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-5f39b2ca-4f13-4346-b14c-8647612e4f1c req-887fca6b-3ee5-42a9-869b-725d134d883c service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:49.837977 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-5f39b2ca-4f13-4346-b14c-8647612e4f1c req-887fca6b-3ee5-42a9-869b-725d134d883c service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:49.838904 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-5f39b2ca-4f13-4346-b14c-8647612e4f1c req-887fca6b-3ee5-42a9-869b-725d134d883c service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:49.862307 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-5f39b2ca-4f13-4346-b14c-8647612e4f1c req-887fca6b-3ee5-42a9-869b-725d134d883c service nova] Creating event network-vif-plugged:1e55ecc3-f194-4529-b75c-be5293dba5c9 for instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 on hv-cinder-maste Aug 21 01:27:49.875115 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-5f39b2ca-4f13-4346-b14c-8647612e4f1c req-887fca6b-3ee5-42a9-869b-725d134d883c service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074968 Aug 21 01:27:49.888376 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 169/343] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:27:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:27:49.977695 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-adcd2952-f08c-4cec-9ec7-4746d2e89455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:49.978773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-adcd2952-f08c-4cec-9ec7-4746d2e89455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:49.990002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-adcd2952-f08c-4cec-9ec7-4746d2e89455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:49.992209 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-adcd2952-f08c-4cec-9ec7-4746d2e89455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:49.993999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-adcd2952-f08c-4cec-9ec7-4746d2e89455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:49.995601 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-adcd2952-f08c-4cec-9ec7-4746d2e89455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:49.997372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-adcd2952-f08c-4cec-9ec7-4746d2e89455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:49.998698 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-adcd2952-f08c-4cec-9ec7-4746d2e89455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:50.124056 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-52b4b327-51a2-4ec9-95f6-f41fe4a699bb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:50.126476 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-52b4b327-51a2-4ec9-95f6-f41fe4a699bb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:50.139812 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-52b4b327-51a2-4ec9-95f6-f41fe4a699bb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:50.142255 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-52b4b327-51a2-4ec9-95f6-f41fe4a699bb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:50.142383 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-52b4b327-51a2-4ec9-95f6-f41fe4a699bb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:50.143760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-52b4b327-51a2-4ec9-95f6-f41fe4a699bb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:50.144726 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-52b4b327-51a2-4ec9-95f6-f41fe4a699bb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:50.151970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-52b4b327-51a2-4ec9-95f6-f41fe4a699bb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:50.963860 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-52b4b327-51a2-4ec9-95f6-f41fe4a699bb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:50.967235 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-adcd2952-f08c-4cec-9ec7-4746d2e89455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:50.984171 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-adcd2952-f08c-4cec-9ec7-4746d2e89455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:50.985617 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-52b4b327-51a2-4ec9-95f6-f41fe4a699bb tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1711 microversion: 2.1 time: 0.869866 Aug 21 01:27:50.986783 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 170/344] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:50 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1711 bytes in 872 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:50.990514 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-adcd2952-f08c-4cec-9ec7-4746d2e89455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:50.995741 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8e569134-a5b7-4619-b9b3-1ef749c6bd7a tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:50.997855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-adcd2952-f08c-4cec-9ec7-4746d2e89455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 1.027406 Aug 21 01:27:51.012637 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 175/345] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:49 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 1043 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:51.018505 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8e569134-a5b7-4619-b9b3-1ef749c6bd7a tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:51.026278 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-53008c1b-de6b-4c35-a3cf-0b2941ac04f8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:51.027167 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-53008c1b-de6b-4c35-a3cf-0b2941ac04f8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:51.029041 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e569134-a5b7-4619-b9b3-1ef749c6bd7a tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:51.031707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e569134-a5b7-4619-b9b3-1ef749c6bd7a tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:51.033986 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e569134-a5b7-4619-b9b3-1ef749c6bd7a tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:51.034925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e569134-a5b7-4619-b9b3-1ef749c6bd7a tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:51.038098 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e569134-a5b7-4619-b9b3-1ef749c6bd7a tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:51.039151 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e569134-a5b7-4619-b9b3-1ef749c6bd7a tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:51.041149 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53008c1b-de6b-4c35-a3cf-0b2941ac04f8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:51.046599 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53008c1b-de6b-4c35-a3cf-0b2941ac04f8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:51.047306 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53008c1b-de6b-4c35-a3cf-0b2941ac04f8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:51.048347 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53008c1b-de6b-4c35-a3cf-0b2941ac04f8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:51.048963 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53008c1b-de6b-4c35-a3cf-0b2941ac04f8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:51.049815 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53008c1b-de6b-4c35-a3cf-0b2941ac04f8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:51.662877 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-53008c1b-de6b-4c35-a3cf-0b2941ac04f8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:51.684199 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-53008c1b-de6b-4c35-a3cf-0b2941ac04f8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.669709 Aug 21 01:27:51.688907 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 176/346] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:51 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 675 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:51.746745 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8e569134-a5b7-4619-b9b3-1ef749c6bd7a tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:51.762417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8e569134-a5b7-4619-b9b3-1ef749c6bd7a tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1314 microversion: 2.1 time: 0.774281 Aug 21 01:27:51.763679 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 171/347] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:50 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1314 bytes in 776 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:52.014081 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5810af08-146f-448d-b369-435601a0a245 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:52.015800 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5810af08-146f-448d-b369-435601a0a245 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:52.026291 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b5396d00-8a12-436d-b09a-58c843e3ac99 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:52.027107 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-b5396d00-8a12-436d-b09a-58c843e3ac99 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:52.039897 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5810af08-146f-448d-b369-435601a0a245 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:52.041863 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b5396d00-8a12-436d-b09a-58c843e3ac99 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:52.044797 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b5396d00-8a12-436d-b09a-58c843e3ac99 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:52.049042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b5396d00-8a12-436d-b09a-58c843e3ac99 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:52.050269 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b5396d00-8a12-436d-b09a-58c843e3ac99 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:52.050952 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5810af08-146f-448d-b369-435601a0a245 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:52.051624 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b5396d00-8a12-436d-b09a-58c843e3ac99 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:52.057527 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b5396d00-8a12-436d-b09a-58c843e3ac99 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:52.059693 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5810af08-146f-448d-b369-435601a0a245 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:52.060675 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5810af08-146f-448d-b369-435601a0a245 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:52.061165 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5810af08-146f-448d-b369-435601a0a245 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:52.061607 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5810af08-146f-448d-b369-435601a0a245 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:52.469167 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5810af08-146f-448d-b369-435601a0a245 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:52.499335 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5810af08-146f-448d-b369-435601a0a245 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1711 microversion: 2.1 time: 0.486462 Aug 21 01:27:52.513980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b5396d00-8a12-436d-b09a-58c843e3ac99 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:52.517973 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 177/348] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:52 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1711 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:52.556770 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b5396d00-8a12-436d-b09a-58c843e3ac99 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:52.560154 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b5396d00-8a12-436d-b09a-58c843e3ac99 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:52.580471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b5396d00-8a12-436d-b09a-58c843e3ac99 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.548234 Aug 21 01:27:52.581707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 172/349] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:52 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:52.772933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a2aded2c-ed24-4cf7-b186-1fc322c03c93 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:52.812371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a2aded2c-ed24-4cf7-b186-1fc322c03c93 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:52.812545 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b66a7cbe-d478-4562-86e1-00722603e08c tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:52.812545 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-b66a7cbe-d478-4562-86e1-00722603e08c tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:52.819213 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a2aded2c-ed24-4cf7-b186-1fc322c03c93 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:52.827322 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a2aded2c-ed24-4cf7-b186-1fc322c03c93 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:52.828443 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b66a7cbe-d478-4562-86e1-00722603e08c tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:52.833663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a2aded2c-ed24-4cf7-b186-1fc322c03c93 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:52.838114 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b66a7cbe-d478-4562-86e1-00722603e08c tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:52.839362 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b66a7cbe-d478-4562-86e1-00722603e08c tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:52.840593 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a2aded2c-ed24-4cf7-b186-1fc322c03c93 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:52.841746 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a2aded2c-ed24-4cf7-b186-1fc322c03c93 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:52.862898 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a2aded2c-ed24-4cf7-b186-1fc322c03c93 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:52.869493 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b66a7cbe-d478-4562-86e1-00722603e08c tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:52.871164 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b66a7cbe-d478-4562-86e1-00722603e08c tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.028s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:52.875587 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b66a7cbe-d478-4562-86e1-00722603e08c tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:53.491450 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a2aded2c-ed24-4cf7-b186-1fc322c03c93 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:53.501468 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b66a7cbe-d478-4562-86e1-00722603e08c tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:53.503466 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a2aded2c-ed24-4cf7-b186-1fc322c03c93 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1370 microversion: 2.1 time: 0.735147 Aug 21 01:27:53.504071 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 178/350] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:52 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1370 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:53.523208 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b66a7cbe-d478-4562-86e1-00722603e08c tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1314 microversion: 2.1 time: 0.737120 Aug 21 01:27:53.523800 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 173/351] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:52 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1314 bytes in 739 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:53.541721 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-cff7e546-e07c-4dd0-ae6c-7f9c35d84ee9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:53.542775 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-cff7e546-e07c-4dd0-ae6c-7f9c35d84ee9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:53.553570 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cff7e546-e07c-4dd0-ae6c-7f9c35d84ee9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:53.554216 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cff7e546-e07c-4dd0-ae6c-7f9c35d84ee9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:53.554821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cff7e546-e07c-4dd0-ae6c-7f9c35d84ee9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:53.555735 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cff7e546-e07c-4dd0-ae6c-7f9c35d84ee9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:53.556471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cff7e546-e07c-4dd0-ae6c-7f9c35d84ee9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:53.557099 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cff7e546-e07c-4dd0-ae6c-7f9c35d84ee9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:53.611352 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-28e8faa7-ad01-4126-9ca9-1ed04cf7195d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:53.612377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-28e8faa7-ad01-4126-9ca9-1ed04cf7195d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:53.641668 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28e8faa7-ad01-4126-9ca9-1ed04cf7195d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:53.642547 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28e8faa7-ad01-4126-9ca9-1ed04cf7195d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:53.644394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28e8faa7-ad01-4126-9ca9-1ed04cf7195d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:53.646639 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28e8faa7-ad01-4126-9ca9-1ed04cf7195d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:53.650033 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28e8faa7-ad01-4126-9ca9-1ed04cf7195d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:53.651268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28e8faa7-ad01-4126-9ca9-1ed04cf7195d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:53.933971 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-cff7e546-e07c-4dd0-ae6c-7f9c35d84ee9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:53.954430 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-cff7e546-e07c-4dd0-ae6c-7f9c35d84ee9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1711 microversion: 2.1 time: 0.412541 Aug 21 01:27:53.954430 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 179/352] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:53 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1711 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:53.971486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-28e8faa7-ad01-4126-9ca9-1ed04cf7195d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:53.982344 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-28e8faa7-ad01-4126-9ca9-1ed04cf7195d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:53.983997 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-28e8faa7-ad01-4126-9ca9-1ed04cf7195d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:53.985420 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-28e8faa7-ad01-4126-9ca9-1ed04cf7195d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.382621 Aug 21 01:27:53.986489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 174/353] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:53 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:54.268496 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-9d57dff3-cf3c-4328-b743-7f1d16210e14 req-68635f6e-8d86-4e54-aa98-bcad8b84c5b4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "09d5a793-bbb6-4169-b790-835069573e2a", "name": "network-vif-plugged", "status": "completed", "tag": "1dad426a-d53f-4f11-9c48-ba0aa747d6c3"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:54.306567 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9d57dff3-cf3c-4328-b743-7f1d16210e14 req-68635f6e-8d86-4e54-aa98-bcad8b84c5b4 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:54.310092 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9d57dff3-cf3c-4328-b743-7f1d16210e14 req-68635f6e-8d86-4e54-aa98-bcad8b84c5b4 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:54.310914 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9d57dff3-cf3c-4328-b743-7f1d16210e14 req-68635f6e-8d86-4e54-aa98-bcad8b84c5b4 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:54.325099 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-9d57dff3-cf3c-4328-b743-7f1d16210e14 req-68635f6e-8d86-4e54-aa98-bcad8b84c5b4 service nova] Creating event network-vif-plugged:1dad426a-d53f-4f11-9c48-ba0aa747d6c3 for instance 09d5a793-bbb6-4169-b790-835069573e2a on hv-cinder-maste Aug 21 01:27:54.332306 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-9d57dff3-cf3c-4328-b743-7f1d16210e14 req-68635f6e-8d86-4e54-aa98-bcad8b84c5b4 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071440 Aug 21 01:27:54.338372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 180/354] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:27:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:27:54.536829 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-44f95ac6-4c9e-4ce1-a3dc-fca5bc243412 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:54.538529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-44f95ac6-4c9e-4ce1-a3dc-fca5bc243412 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:54.554568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-77c3aff7-614f-46f0-a34d-41a64a9ff7b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:54.556321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-77c3aff7-614f-46f0-a34d-41a64a9ff7b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:54.588620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-77c3aff7-614f-46f0-a34d-41a64a9ff7b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:54.589115 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-77c3aff7-614f-46f0-a34d-41a64a9ff7b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:54.589548 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-77c3aff7-614f-46f0-a34d-41a64a9ff7b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:54.590248 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-77c3aff7-614f-46f0-a34d-41a64a9ff7b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:54.590963 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-77c3aff7-614f-46f0-a34d-41a64a9ff7b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:54.591181 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-77c3aff7-614f-46f0-a34d-41a64a9ff7b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:54.593941 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-44f95ac6-4c9e-4ce1-a3dc-fca5bc243412 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:54.594733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-44f95ac6-4c9e-4ce1-a3dc-fca5bc243412 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:54.595348 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-44f95ac6-4c9e-4ce1-a3dc-fca5bc243412 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:54.596602 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-44f95ac6-4c9e-4ce1-a3dc-fca5bc243412 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:54.597148 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-44f95ac6-4c9e-4ce1-a3dc-fca5bc243412 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:54.597597 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-44f95ac6-4c9e-4ce1-a3dc-fca5bc243412 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:54.997024 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-44f95ac6-4c9e-4ce1-a3dc-fca5bc243412 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:55.016299 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-44f95ac6-4c9e-4ce1-a3dc-fca5bc243412 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1563 microversion: 2.1 time: 0.489310 Aug 21 01:27:55.032827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 175/355] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:54 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1563 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:55.040091 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d01a50a6-5205-48b4-a655-0599ec9c5cc3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:55.045191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d01a50a6-5205-48b4-a655-0599ec9c5cc3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:55.068653 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d01a50a6-5205-48b4-a655-0599ec9c5cc3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:55.069986 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d01a50a6-5205-48b4-a655-0599ec9c5cc3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:55.070593 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d01a50a6-5205-48b4-a655-0599ec9c5cc3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:55.071606 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d01a50a6-5205-48b4-a655-0599ec9c5cc3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:55.072273 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d01a50a6-5205-48b4-a655-0599ec9c5cc3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:55.078612 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d01a50a6-5205-48b4-a655-0599ec9c5cc3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:55.081631 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-77c3aff7-614f-46f0-a34d-41a64a9ff7b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:55.098900 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-77c3aff7-614f-46f0-a34d-41a64a9ff7b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1314 microversion: 2.1 time: 0.554363 Aug 21 01:27:55.100150 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 181/356] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:54 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1314 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:55.114662 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4679c549-fd72-4fdc-b890-0a3be37bcbb8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:55.115749 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4679c549-fd72-4fdc-b890-0a3be37bcbb8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:55.122941 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4679c549-fd72-4fdc-b890-0a3be37bcbb8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:55.123819 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4679c549-fd72-4fdc-b890-0a3be37bcbb8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:55.124585 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4679c549-fd72-4fdc-b890-0a3be37bcbb8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:55.125606 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4679c549-fd72-4fdc-b890-0a3be37bcbb8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:55.126287 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4679c549-fd72-4fdc-b890-0a3be37bcbb8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:55.126940 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4679c549-fd72-4fdc-b890-0a3be37bcbb8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:55.456972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4679c549-fd72-4fdc-b890-0a3be37bcbb8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:55.477507 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4679c549-fd72-4fdc-b890-0a3be37bcbb8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:55.480672 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4679c549-fd72-4fdc-b890-0a3be37bcbb8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:55.499956 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4679c549-fd72-4fdc-b890-0a3be37bcbb8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.379771 Aug 21 01:27:55.499956 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 182/357] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:55 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:55.499956 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-40fb6efd-f922-41fe-995f-f967ae046964 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:55.499956 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-40fb6efd-f922-41fe-995f-f967ae046964 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:55.501867 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-40fb6efd-f922-41fe-995f-f967ae046964 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:55.503548 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-40fb6efd-f922-41fe-995f-f967ae046964 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:55.504221 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-40fb6efd-f922-41fe-995f-f967ae046964 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:55.504963 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-40fb6efd-f922-41fe-995f-f967ae046964 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:55.505416 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-40fb6efd-f922-41fe-995f-f967ae046964 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:55.505843 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-40fb6efd-f922-41fe-995f-f967ae046964 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:55.511282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d01a50a6-5205-48b4-a655-0599ec9c5cc3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:55.525251 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d01a50a6-5205-48b4-a655-0599ec9c5cc3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1715 microversion: 2.1 time: 0.490226 Aug 21 01:27:55.536764 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 176/358] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:55 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1715 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:55.683758 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-40fb6efd-f922-41fe-995f-f967ae046964 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.199263 Aug 21 01:27:55.684401 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 183/359] 10.222.0.47 () {58 vars in 1275 bytes} [Sun Aug 21 01:27:55 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-interface => generated 284 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:27:56.121688 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d7b88805-1791-4dcd-9fe9-ac6e71bfadce tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:56.122935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d7b88805-1791-4dcd-9fe9-ac6e71bfadce tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:56.138113 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d7b88805-1791-4dcd-9fe9-ac6e71bfadce tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:56.138447 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d7b88805-1791-4dcd-9fe9-ac6e71bfadce tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:56.139101 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d7b88805-1791-4dcd-9fe9-ac6e71bfadce tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:56.139878 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d7b88805-1791-4dcd-9fe9-ac6e71bfadce tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:56.140309 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d7b88805-1791-4dcd-9fe9-ac6e71bfadce tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:56.140919 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d7b88805-1791-4dcd-9fe9-ac6e71bfadce tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:56.462573 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d7b88805-1791-4dcd-9fe9-ac6e71bfadce tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:56.489435 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d7b88805-1791-4dcd-9fe9-ac6e71bfadce tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1314 microversion: 2.1 time: 0.372339 Aug 21 01:27:56.501776 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 177/360] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:56 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1314 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:56.515047 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8d851805-8b6c-432f-8472-a85a0d6cfc29 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:56.515684 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8d851805-8b6c-432f-8472-a85a0d6cfc29 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:56.525497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d851805-8b6c-432f-8472-a85a0d6cfc29 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:56.556292 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d851805-8b6c-432f-8472-a85a0d6cfc29 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:56.556292 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d851805-8b6c-432f-8472-a85a0d6cfc29 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:56.556292 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d851805-8b6c-432f-8472-a85a0d6cfc29 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:56.556292 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d851805-8b6c-432f-8472-a85a0d6cfc29 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:56.556837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d851805-8b6c-432f-8472-a85a0d6cfc29 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:56.782104 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8d851805-8b6c-432f-8472-a85a0d6cfc29 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:56.824203 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8d851805-8b6c-432f-8472-a85a0d6cfc29 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:56.826588 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8d851805-8b6c-432f-8472-a85a0d6cfc29 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:56.828065 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8d851805-8b6c-432f-8472-a85a0d6cfc29 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.324558 Aug 21 01:27:56.829022 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 184/361] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:56 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:57.490236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-83198270-1d2e-4826-87a1-6220c88b26cb tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Action: 'create', calling method: .version_select at 0x7f9da3df5040>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1001539393", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE+JGgwTlXpHFNafuDEvtL0FxSPumBm3+F1LDC0FCzhkUmlmnOx/JpV0rbLII0b3h7AdQlvJx4uZ5xLPkwzqkEuH/Yihtc7607RrSLodchq0Xte+WqBOLyHz9x5n9mvRuA=="}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:57.519077 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-22e8374d-1fd3-4a3e-813b-ac1e41bc0c6d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:57.520077 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-22e8374d-1fd3-4a3e-813b-ac1e41bc0c6d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:57.556727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22e8374d-1fd3-4a3e-813b-ac1e41bc0c6d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:57.556727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22e8374d-1fd3-4a3e-813b-ac1e41bc0c6d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:57.571362 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22e8374d-1fd3-4a3e-813b-ac1e41bc0c6d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:57.577054 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22e8374d-1fd3-4a3e-813b-ac1e41bc0c6d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:57.577552 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22e8374d-1fd3-4a3e-813b-ac1e41bc0c6d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:57.577994 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22e8374d-1fd3-4a3e-813b-ac1e41bc0c6d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:57.615175 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-83198270-1d2e-4826-87a1-6220c88b26cb tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Getting quotas for project da8fc458ef6b49e18daac9fb195a2149. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:27:57.629879 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-83198270-1d2e-4826-87a1-6220c88b26cb tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Getting quotas for user 2a6687a3b09445d897879095287a8586 and project da8fc458ef6b49e18daac9fb195a2149. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:27:57.879349 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-83198270-1d2e-4826-87a1-6220c88b26cb tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.393340 Aug 21 01:27:57.880119 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 178/362] 10.222.0.47 () {60 vars in 1159 bytes} [Sun Aug 21 01:27:57 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:27:57.900753 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-af1962e1-c0a1-4870-8f82-715a0519cb7d req-5eeb2dfd-17e5-4bde-809f-5424f8145f35 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eddb6b3f-8ca0-4f71-b371-19c6e65dd095", "tag": "1e55ecc3-f194-4529-b75c-be5293dba5c9"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:57.919080 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-af1962e1-c0a1-4870-8f82-715a0519cb7d req-5eeb2dfd-17e5-4bde-809f-5424f8145f35 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:57.920891 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-af1962e1-c0a1-4870-8f82-715a0519cb7d req-5eeb2dfd-17e5-4bde-809f-5424f8145f35 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:57.921588 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-af1962e1-c0a1-4870-8f82-715a0519cb7d req-5eeb2dfd-17e5-4bde-809f-5424f8145f35 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:57.943294 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-af1962e1-c0a1-4870-8f82-715a0519cb7d req-5eeb2dfd-17e5-4bde-809f-5424f8145f35 service nova] Creating event network-changed:1e55ecc3-f194-4529-b75c-be5293dba5c9 for instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 on hv-cinder-maste Aug 21 01:27:57.955463 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-af1962e1-c0a1-4870-8f82-715a0519cb7d req-5eeb2dfd-17e5-4bde-809f-5424f8145f35 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070462 Aug 21 01:27:57.956934 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 179/363] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:27:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:27:57.960587 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-22e8374d-1fd3-4a3e-813b-ac1e41bc0c6d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:57.965795 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-40264b73-a60e-4d82-9ad0-ea5503bb49b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:57.969712 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-40264b73-a60e-4d82-9ad0-ea5503bb49b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:57.976733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-40264b73-a60e-4d82-9ad0-ea5503bb49b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:57.977625 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-40264b73-a60e-4d82-9ad0-ea5503bb49b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:57.978282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-40264b73-a60e-4d82-9ad0-ea5503bb49b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:57.979591 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-40264b73-a60e-4d82-9ad0-ea5503bb49b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:57.980645 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-40264b73-a60e-4d82-9ad0-ea5503bb49b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:57.981430 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-40264b73-a60e-4d82-9ad0-ea5503bb49b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:57.983339 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-22e8374d-1fd3-4a3e-813b-ac1e41bc0c6d tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1463 microversion: 2.1 time: 0.468377 Aug 21 01:27:58.004438 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 185/364] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:57 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1463 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:27:58.051860 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-53436dbc-d732-4d13-8312-8dfa5beda652 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:58.052916 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-53436dbc-d732-4d13-8312-8dfa5beda652 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:58.068062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53436dbc-d732-4d13-8312-8dfa5beda652 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:58.068596 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53436dbc-d732-4d13-8312-8dfa5beda652 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:58.070530 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53436dbc-d732-4d13-8312-8dfa5beda652 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:58.071506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53436dbc-d732-4d13-8312-8dfa5beda652 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:58.071974 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53436dbc-d732-4d13-8312-8dfa5beda652 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:58.074142 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53436dbc-d732-4d13-8312-8dfa5beda652 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:58.125368 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-53436dbc-d732-4d13-8312-8dfa5beda652 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Going to try to terminate instance {{(pid=121277) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:27:58.315502 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-40264b73-a60e-4d82-9ad0-ea5503bb49b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:58.327795 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-40264b73-a60e-4d82-9ad0-ea5503bb49b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:58.332706 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-40264b73-a60e-4d82-9ad0-ea5503bb49b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:58.334790 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-40264b73-a60e-4d82-9ad0-ea5503bb49b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.376062 Aug 21 01:27:58.356301 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 180/365] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:57 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:27:58.362473 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-74976085", "imageRef": "c34b2489-4734-4c13-b123-ab85011296c7", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1001539393", "networks": [{"uuid": "1e79e20e-bb84-4864-9fe8-d17ca136af81"}]}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:27:58.429182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] validate_networks() for [('1e79e20e-bb84-4864-9fe8-d17ca136af81', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:27:58.989890 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:27:58.992494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:27:58.992494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:27:58.992494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:27:58.992494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:27:59.001026 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:59.001694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:59.001963 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:59.002705 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:59.003164 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:59.003594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:59.015657 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:59.023805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Getting quotas for project da8fc458ef6b49e18daac9fb195a2149. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:27:59.028619 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Getting quotas for user 2a6687a3b09445d897879095287a8586 and project da8fc458ef6b49e18daac9fb195a2149. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:27:59.045763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:27:59.048094 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c34b2489-4734-4c13-b123-ab85011296c7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:27:59.271816 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-53436dbc-d732-4d13-8312-8dfa5beda652 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "DELETE /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 204 len: 0 microversion: 2.1 time: 1.225671 Aug 21 01:27:59.279479 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 186/366] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:27:58 2022] DELETE /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 0 bytes in 1229 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:27:59.287285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-0d86553c-1511-4c06-b15d-f467d207c26e tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:59.288008 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-0d86553c-1511-4c06-b15d-f467d207c26e tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:59.295369 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d86553c-1511-4c06-b15d-f467d207c26e tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:59.296275 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d86553c-1511-4c06-b15d-f467d207c26e tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:59.296997 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d86553c-1511-4c06-b15d-f467d207c26e tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:59.298894 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d86553c-1511-4c06-b15d-f467d207c26e tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:59.299503 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d86553c-1511-4c06-b15d-f467d207c26e tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:59.300251 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d86553c-1511-4c06-b15d-f467d207c26e tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:59.308790 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:27:59.421766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ecb43b22-6ac7-4916-b8c6-a56cd0f81713 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.063511 Aug 21 01:27:59.422754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 181/367] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:27:58 2022] POST /compute/v2.1/servers => generated 380 bytes in 1065 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:27:59.443376 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f0b6930d-bbdc-4c42-afd9-838c83449729 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:59.443376 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f0b6930d-bbdc-4c42-afd9-838c83449729 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:59.446275 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f0b6930d-bbdc-4c42-afd9-838c83449729 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:59.447110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f0b6930d-bbdc-4c42-afd9-838c83449729 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:59.447751 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f0b6930d-bbdc-4c42-afd9-838c83449729 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:59.448720 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f0b6930d-bbdc-4c42-afd9-838c83449729 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:27:59.449367 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f0b6930d-bbdc-4c42-afd9-838c83449729 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:27:59.449970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f0b6930d-bbdc-4c42-afd9-838c83449729 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:27:59.516221 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-0d86553c-1511-4c06-b15d-f467d207c26e tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:59.528745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-0d86553c-1511-4c06-b15d-f467d207c26e tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1469 microversion: 2.1 time: 0.250122 Aug 21 01:27:59.535501 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 187/368] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:59 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1469 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:59.544318 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3a3a1d3e-f849-4450-ab5c-99bce30b2e19 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:27:59.549107 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3a3a1d3e-f849-4450-ab5c-99bce30b2e19 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:27:59.689321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3a3a1d3e-f849-4450-ab5c-99bce30b2e19 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:59.700365 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3a3a1d3e-f849-4450-ab5c-99bce30b2e19 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1257 microversion: 2.1 time: 0.163245 Aug 21 01:27:59.717010 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 188/369] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:27:59 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1257 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:27:59.723464 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f0b6930d-bbdc-4c42-afd9-838c83449729 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:59.740222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f0b6930d-bbdc-4c42-afd9-838c83449729 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:59.740222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f0b6930d-bbdc-4c42-afd9-838c83449729 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:27:59.740222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f0b6930d-bbdc-4c42-afd9-838c83449729 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.315999 Aug 21 01:27:59.741013 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 182/370] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:27:59 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:00.557273 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e7f30c6a-1071-4789-8e46-d94560b3ffa3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:00.586999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e7f30c6a-1071-4789-8e46-d94560b3ffa3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:00.586999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e7f30c6a-1071-4789-8e46-d94560b3ffa3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:00.586999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e7f30c6a-1071-4789-8e46-d94560b3ffa3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:00.586999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e7f30c6a-1071-4789-8e46-d94560b3ffa3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:00.587433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e7f30c6a-1071-4789-8e46-d94560b3ffa3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:00.587433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e7f30c6a-1071-4789-8e46-d94560b3ffa3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:00.587433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e7f30c6a-1071-4789-8e46-d94560b3ffa3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:00.723603 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c428b687-400f-4944-9341-28083823d6c5 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:00.725437 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c428b687-400f-4944-9341-28083823d6c5 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:00.731299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e7f30c6a-1071-4789-8e46-d94560b3ffa3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:00.749081 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e7f30c6a-1071-4789-8e46-d94560b3ffa3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1469 microversion: 2.1 time: 0.196228 Aug 21 01:28:00.749995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 189/371] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:00 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1469 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:00.769354 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c01f5a0a-58d1-48c8-9787-a5ab2943ecbc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:00.770425 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c01f5a0a-58d1-48c8-9787-a5ab2943ecbc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:00.778944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c01f5a0a-58d1-48c8-9787-a5ab2943ecbc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:00.779606 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c01f5a0a-58d1-48c8-9787-a5ab2943ecbc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:00.780039 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c01f5a0a-58d1-48c8-9787-a5ab2943ecbc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:00.782256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c01f5a0a-58d1-48c8-9787-a5ab2943ecbc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:00.782256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c01f5a0a-58d1-48c8-9787-a5ab2943ecbc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:00.782256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c01f5a0a-58d1-48c8-9787-a5ab2943ecbc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:00.795241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c428b687-400f-4944-9341-28083823d6c5 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:00.806282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c428b687-400f-4944-9341-28083823d6c5 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1257 microversion: 2.1 time: 0.088038 Aug 21 01:28:00.820967 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 183/372] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:00 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1257 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:00.962258 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c01f5a0a-58d1-48c8-9787-a5ab2943ecbc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:00.971348 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c01f5a0a-58d1-48c8-9787-a5ab2943ecbc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:00.972937 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c01f5a0a-58d1-48c8-9787-a5ab2943ecbc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:00.974097 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c01f5a0a-58d1-48c8-9787-a5ab2943ecbc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.211986 Aug 21 01:28:00.974864 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 190/373] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:00 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:28:01.771150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-69d47fc5-81e2-4f8d-a76d-dbdebcacd305 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:01.771752 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-69d47fc5-81e2-4f8d-a76d-dbdebcacd305 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:01.780045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-69d47fc5-81e2-4f8d-a76d-dbdebcacd305 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:01.781197 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-69d47fc5-81e2-4f8d-a76d-dbdebcacd305 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:01.781197 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-69d47fc5-81e2-4f8d-a76d-dbdebcacd305 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:01.781755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-69d47fc5-81e2-4f8d-a76d-dbdebcacd305 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:01.782217 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-69d47fc5-81e2-4f8d-a76d-dbdebcacd305 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:01.782642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-69d47fc5-81e2-4f8d-a76d-dbdebcacd305 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:01.830515 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f0ed3a4d-0796-4bb1-acd3-7c3acae530ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:01.832908 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f0ed3a4d-0796-4bb1-acd3-7c3acae530ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:01.838915 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f0ed3a4d-0796-4bb1-acd3-7c3acae530ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:01.839408 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f0ed3a4d-0796-4bb1-acd3-7c3acae530ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:01.839863 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f0ed3a4d-0796-4bb1-acd3-7c3acae530ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:01.840626 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f0ed3a4d-0796-4bb1-acd3-7c3acae530ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:01.842009 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f0ed3a4d-0796-4bb1-acd3-7c3acae530ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:01.842009 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f0ed3a4d-0796-4bb1-acd3-7c3acae530ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:01.922629 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f0ed3a4d-0796-4bb1-acd3-7c3acae530ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:01.930908 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f0ed3a4d-0796-4bb1-acd3-7c3acae530ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1261 microversion: 2.1 time: 0.104343 Aug 21 01:28:01.931497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 191/374] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:01 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1261 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:01.952825 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-69d47fc5-81e2-4f8d-a76d-dbdebcacd305 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:01.962322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-69d47fc5-81e2-4f8d-a76d-dbdebcacd305 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1469 microversion: 2.1 time: 0.195113 Aug 21 01:28:01.962891 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 184/375] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:01 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1469 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:01.997334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b1e5a0d6-fa56-4d4c-ac00-c8c26400ecb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:01.997947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b1e5a0d6-fa56-4d4c-ac00-c8c26400ecb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:02.004045 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b1e5a0d6-fa56-4d4c-ac00-c8c26400ecb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:02.004559 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b1e5a0d6-fa56-4d4c-ac00-c8c26400ecb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:02.004993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b1e5a0d6-fa56-4d4c-ac00-c8c26400ecb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:02.005683 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b1e5a0d6-fa56-4d4c-ac00-c8c26400ecb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:02.006140 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b1e5a0d6-fa56-4d4c-ac00-c8c26400ecb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:02.006563 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b1e5a0d6-fa56-4d4c-ac00-c8c26400ecb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:02.144486 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b1e5a0d6-fa56-4d4c-ac00-c8c26400ecb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:02.153561 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b1e5a0d6-fa56-4d4c-ac00-c8c26400ecb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:02.155104 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b1e5a0d6-fa56-4d4c-ac00-c8c26400ecb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:02.156291 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b1e5a0d6-fa56-4d4c-ac00-c8c26400ecb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.162985 Aug 21 01:28:02.156853 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 192/376] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:01 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:02.951517 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a54605a5-8336-493f-9b2e-0a1c5a92149f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:02.952318 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a54605a5-8336-493f-9b2e-0a1c5a92149f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:02.959078 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a54605a5-8336-493f-9b2e-0a1c5a92149f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:02.959557 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a54605a5-8336-493f-9b2e-0a1c5a92149f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:02.959989 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a54605a5-8336-493f-9b2e-0a1c5a92149f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:02.961245 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a54605a5-8336-493f-9b2e-0a1c5a92149f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:02.961695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a54605a5-8336-493f-9b2e-0a1c5a92149f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:02.962116 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a54605a5-8336-493f-9b2e-0a1c5a92149f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:02.984848 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-1d218fd5-b793-4d19-94ee-651e83c60acd tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:02.988595 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-1d218fd5-b793-4d19-94ee-651e83c60acd tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:02.991700 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1d218fd5-b793-4d19-94ee-651e83c60acd tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:02.992216 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1d218fd5-b793-4d19-94ee-651e83c60acd tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:02.992764 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1d218fd5-b793-4d19-94ee-651e83c60acd tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:02.993481 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1d218fd5-b793-4d19-94ee-651e83c60acd tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:02.993943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1d218fd5-b793-4d19-94ee-651e83c60acd tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:02.994369 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1d218fd5-b793-4d19-94ee-651e83c60acd tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:03.033295 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a54605a5-8336-493f-9b2e-0a1c5a92149f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:03.037282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=100 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:03.043224 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a54605a5-8336-493f-9b2e-0a1c5a92149f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1261 microversion: 2.1 time: 0.095975 Aug 21 01:28:03.043815 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 185/377] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:02 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1261 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:03.137273 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-1d218fd5-b793-4d19-94ee-651e83c60acd tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:03.145223 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-1d218fd5-b793-4d19-94ee-651e83c60acd tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1469 microversion: 2.1 time: 0.164249 Aug 21 01:28:03.145806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 193/378] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:02 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1469 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:03.183230 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-eec416e4-cce6-44b1-b27d-7ee523d3803a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:03.183230 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-eec416e4-cce6-44b1-b27d-7ee523d3803a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:03.187880 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eec416e4-cce6-44b1-b27d-7ee523d3803a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:03.188391 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eec416e4-cce6-44b1-b27d-7ee523d3803a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:03.188891 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eec416e4-cce6-44b1-b27d-7ee523d3803a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:03.189571 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eec416e4-cce6-44b1-b27d-7ee523d3803a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:03.190022 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eec416e4-cce6-44b1-b27d-7ee523d3803a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:03.190444 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eec416e4-cce6-44b1-b27d-7ee523d3803a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:03.348774 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-eec416e4-cce6-44b1-b27d-7ee523d3803a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:03.355322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-eec416e4-cce6-44b1-b27d-7ee523d3803a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:03.357333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-eec416e4-cce6-44b1-b27d-7ee523d3803a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:03.358101 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-eec416e4-cce6-44b1-b27d-7ee523d3803a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1738 microversion: 2.20 time: 0.181864 Aug 21 01:28:03.358675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 186/379] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:03 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1738 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:03.805765 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-afd040ca-ac4d-4040-a457-f83dc08816bb req-120bc907-3627-45a8-bb46-a2d5b10afd01 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "09d5a793-bbb6-4169-b790-835069573e2a", "name": "network-vif-deleted", "tag": "1dad426a-d53f-4f11-9c48-ba0aa747d6c3"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:03.815822 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-afd040ca-ac4d-4040-a457-f83dc08816bb req-120bc907-3627-45a8-bb46-a2d5b10afd01 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:03.816323 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-afd040ca-ac4d-4040-a457-f83dc08816bb req-120bc907-3627-45a8-bb46-a2d5b10afd01 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:03.816759 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-afd040ca-ac4d-4040-a457-f83dc08816bb req-120bc907-3627-45a8-bb46-a2d5b10afd01 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:03.829807 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-afd040ca-ac4d-4040-a457-f83dc08816bb req-120bc907-3627-45a8-bb46-a2d5b10afd01 service nova] Creating event network-vif-deleted:1dad426a-d53f-4f11-9c48-ba0aa747d6c3 for instance 09d5a793-bbb6-4169-b790-835069573e2a on hv-cinder-maste Aug 21 01:28:03.835540 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-afd040ca-ac4d-4040-a457-f83dc08816bb req-120bc907-3627-45a8-bb46-a2d5b10afd01 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034588 Aug 21 01:28:03.841060 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 194/380] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:28:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:28:04.064687 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6f223b5f-8f75-4085-9a82-53957aef5f5e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:04.064687 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6f223b5f-8f75-4085-9a82-53957aef5f5e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:04.073521 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6f223b5f-8f75-4085-9a82-53957aef5f5e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:04.074165 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6f223b5f-8f75-4085-9a82-53957aef5f5e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:04.074612 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6f223b5f-8f75-4085-9a82-53957aef5f5e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:04.075347 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6f223b5f-8f75-4085-9a82-53957aef5f5e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:04.076139 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6f223b5f-8f75-4085-9a82-53957aef5f5e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:04.076676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6f223b5f-8f75-4085-9a82-53957aef5f5e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:04.145344 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6f223b5f-8f75-4085-9a82-53957aef5f5e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:04.156512 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6f223b5f-8f75-4085-9a82-53957aef5f5e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1309 microversion: 2.1 time: 0.097155 Aug 21 01:28:04.157138 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 187/381] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:04 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1309 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:04.168577 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ccaf1629-a8ac-4785-b203-7b11848a60f3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:04.173042 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-ccaf1629-a8ac-4785-b203-7b11848a60f3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:04.175771 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ccaf1629-a8ac-4785-b203-7b11848a60f3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:04.176335 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ccaf1629-a8ac-4785-b203-7b11848a60f3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:04.177049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ccaf1629-a8ac-4785-b203-7b11848a60f3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:04.177751 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ccaf1629-a8ac-4785-b203-7b11848a60f3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:04.178210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ccaf1629-a8ac-4785-b203-7b11848a60f3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:04.178635 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ccaf1629-a8ac-4785-b203-7b11848a60f3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:04.257251 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ccaf1629-a8ac-4785-b203-7b11848a60f3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:04.266913 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ccaf1629-a8ac-4785-b203-7b11848a60f3 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1299 microversion: 2.1 time: 0.103526 Aug 21 01:28:04.267551 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 195/382] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:04 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1299 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:04.383800 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4d800d9d-91c4-4586-9fa7-8efb14cf66e4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:04.383800 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4d800d9d-91c4-4586-9fa7-8efb14cf66e4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:04.391491 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4d800d9d-91c4-4586-9fa7-8efb14cf66e4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:04.392188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4d800d9d-91c4-4586-9fa7-8efb14cf66e4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:04.392723 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4d800d9d-91c4-4586-9fa7-8efb14cf66e4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:04.393543 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4d800d9d-91c4-4586-9fa7-8efb14cf66e4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:04.394378 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4d800d9d-91c4-4586-9fa7-8efb14cf66e4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:04.395315 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4d800d9d-91c4-4586-9fa7-8efb14cf66e4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:04.553069 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4d800d9d-91c4-4586-9fa7-8efb14cf66e4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:04.561973 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4d800d9d-91c4-4586-9fa7-8efb14cf66e4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:04.563500 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4d800d9d-91c4-4586-9fa7-8efb14cf66e4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:04.564737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4d800d9d-91c4-4586-9fa7-8efb14cf66e4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1720 microversion: 2.20 time: 0.187139 Aug 21 01:28:04.565363 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 188/383] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:04 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1720 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:05.182053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-339b0de3-0a7d-4c27-b0bd-310cf4a84c2e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:05.182929 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-339b0de3-0a7d-4c27-b0bd-310cf4a84c2e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:05.192652 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-339b0de3-0a7d-4c27-b0bd-310cf4a84c2e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:05.193346 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-339b0de3-0a7d-4c27-b0bd-310cf4a84c2e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:05.199806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-339b0de3-0a7d-4c27-b0bd-310cf4a84c2e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:05.199806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-339b0de3-0a7d-4c27-b0bd-310cf4a84c2e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:05.199806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-339b0de3-0a7d-4c27-b0bd-310cf4a84c2e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:05.199806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-339b0de3-0a7d-4c27-b0bd-310cf4a84c2e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:05.286913 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:05.287739 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:05.298351 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-339b0de3-0a7d-4c27-b0bd-310cf4a84c2e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:05.301327 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:05.302339 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:05.302973 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:05.303853 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:05.304507 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:05.305603 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:05.321392 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-339b0de3-0a7d-4c27-b0bd-310cf4a84c2e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1317 microversion: 2.1 time: 0.143322 Aug 21 01:28:05.323077 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 196/384] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:05 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1317 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:05.343967 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:05.344507 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:05.345113 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:05.345975 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lazy-loading 'fault' on Instance uuid 09d5a793-bbb6-4169-b790-835069573e2a {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:28:05.417097 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '78a9f62090244baa883ca47efed0ae20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e8775cdf1c9c40e8aac986896c4e0b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:05.429260 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2c7f012f-6df9-4150-988e-7fcf9ad1e3b8 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 200 len: 1304 microversion: 2.1 time: 0.145658 Aug 21 01:28:05.432508 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 189/385] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:05 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 1304 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:05.591700 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ff086b46-a918-4bd5-a3ef-1d82b51605d3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:05.592418 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-ff086b46-a918-4bd5-a3ef-1d82b51605d3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:05.598527 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ff086b46-a918-4bd5-a3ef-1d82b51605d3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:05.599170 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ff086b46-a918-4bd5-a3ef-1d82b51605d3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:05.599774 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ff086b46-a918-4bd5-a3ef-1d82b51605d3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:05.600602 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ff086b46-a918-4bd5-a3ef-1d82b51605d3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:05.601060 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ff086b46-a918-4bd5-a3ef-1d82b51605d3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:05.601664 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ff086b46-a918-4bd5-a3ef-1d82b51605d3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:05.830569 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ff086b46-a918-4bd5-a3ef-1d82b51605d3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:05.838837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ff086b46-a918-4bd5-a3ef-1d82b51605d3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:05.840384 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ff086b46-a918-4bd5-a3ef-1d82b51605d3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:05.841544 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ff086b46-a918-4bd5-a3ef-1d82b51605d3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1720 microversion: 2.20 time: 0.253726 Aug 21 01:28:05.842104 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 197/386] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:05 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1720 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:06.118953 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-aa2e622a-5204-4435-811a-3c0ac7404863 req-9e6abf91-59c8-4375-97ab-4147a4144a77 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1", "name": "network-vif-unplugged", "status": "completed", "tag": "9a09b4f3-32e5-4362-82ef-ab73b634b1bf"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:06.130582 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-aa2e622a-5204-4435-811a-3c0ac7404863 req-9e6abf91-59c8-4375-97ab-4147a4144a77 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:06.131122 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-aa2e622a-5204-4435-811a-3c0ac7404863 req-9e6abf91-59c8-4375-97ab-4147a4144a77 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:06.131814 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-aa2e622a-5204-4435-811a-3c0ac7404863 req-9e6abf91-59c8-4375-97ab-4147a4144a77 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:06.144543 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-aa2e622a-5204-4435-811a-3c0ac7404863 req-9e6abf91-59c8-4375-97ab-4147a4144a77 service nova] Creating event network-vif-unplugged:9a09b4f3-32e5-4362-82ef-ab73b634b1bf for instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 on hv-cinder-maste Aug 21 01:28:06.150694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-aa2e622a-5204-4435-811a-3c0ac7404863 req-9e6abf91-59c8-4375-97ab-4147a4144a77 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035309 Aug 21 01:28:06.151367 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 190/387] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:28:06 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:28:06.350615 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-0d0e60e0-2945-4b74-b45c-b6518135138f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:06.351407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-0d0e60e0-2945-4b74-b45c-b6518135138f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:06.358203 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d0e60e0-2945-4b74-b45c-b6518135138f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:06.358692 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d0e60e0-2945-4b74-b45c-b6518135138f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:06.359121 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d0e60e0-2945-4b74-b45c-b6518135138f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:06.359844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d0e60e0-2945-4b74-b45c-b6518135138f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:06.360325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d0e60e0-2945-4b74-b45c-b6518135138f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:06.360803 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d0e60e0-2945-4b74-b45c-b6518135138f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:06.456853 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7d598433-0195-4be8-add4-33515e14c614 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:06.457565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7d598433-0195-4be8-add4-33515e14c614 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] [instance: 09d5a793-bbb6-4169-b790-835069573e2a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:06.465604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7d598433-0195-4be8-add4-33515e14c614 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:06.465604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7d598433-0195-4be8-add4-33515e14c614 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:06.465604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7d598433-0195-4be8-add4-33515e14c614 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:06.466622 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7d598433-0195-4be8-add4-33515e14c614 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:06.466745 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7d598433-0195-4be8-add4-33515e14c614 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:06.467070 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7d598433-0195-4be8-add4-33515e14c614 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:06.483648 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-7d598433-0195-4be8-add4-33515e14c614 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] HTTP exception thrown: Instance 09d5a793-bbb6-4169-b790-835069573e2a could not be found. Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7d598433-0195-4be8-add4-33515e14c614 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Request method failure captured: Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a HTTP/1.1 Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYn_RVg3kVjsFeQ-vuGvKLrB4YaCQfYfz3bx6u49F7AYp41eggETuHuad2aouOn8c9IvORc4Y_fKBHTtDXhkNpE_vDwvpjU7FqDAh3ov2HNmBhMf5DWqyYdNHyWem9bkIaKTtOZ3Xa1DZd7KZAvI5yUGLZlvyDvmOk7IYKyNcDd39HA Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: e8775cdf1c9c40e8aac986896c4e0b01 Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-VolumesNegativeTest-143717074 Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:28:06.512377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:28:06.515019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/e8775cdf1c9c40e8aac986896c4e0b01"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_e8775cdf1c9c40e8aac986896c4e0b01"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/e8775cdf1c9c40e8aac986896c4e0b01"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/e8775cdf1c9c40e8aac986896c4e0b01"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:28:06.515019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-VolumesNegativeTest-143717074 Aug 21 01:28:06.515019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: e8775cdf1c9c40e8aac986896c4e0b01 Aug 21 01:28:06.515019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-VolumesNegativeTest-143717074 Aug 21 01:28:06.515019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-VolumesNegativeTest-143717074-project Aug 21 01:28:06.515019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:28:06.515019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:28:06.515019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: 78a9f62090244baa883ca47efed0ae20 Aug 21 01:28:06.515019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-VolumesNegativeTest-143717074-project Aug 21 01:28:06.515019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:28:06.515019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 09d5a793-bbb6-4169-b790-835069573e2a could not be found. Aug 21 01:28:06.515019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '09d5a793-bbb6-4169-b790-835069573e2a'} Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 09d5a793-bbb6-4169-b790-835069573e2a could not be found. Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:28:06.516158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 09d5a793-bbb6-4169-b790-835069573e2a could not be found. Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 09d5a793-bbb6-4169-b790-835069573e2a could not be found. Aug 21 01:28:06.517694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:28:06.519079 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7d598433-0195-4be8-add4-33515e14c614 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] Returning 404 to user: Instance 09d5a793-bbb6-4169-b790-835069573e2a could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:28:06.519079 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7d598433-0195-4be8-add4-33515e14c614 tempest-VolumesNegativeTest-143717074 tempest-VolumesNegativeTest-143717074-project] 10.222.0.47 "GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a" status: 404 len: 111 microversion: 2.1 time: 0.035021 Aug 21 01:28:06.519079 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 191/388] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:06 2022] GET /compute/v2.1/servers/09d5a793-bbb6-4169-b790-835069573e2a => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:28:06.519334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-0d0e60e0-2945-4b74-b45c-b6518135138f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:06.519334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-0d0e60e0-2945-4b74-b45c-b6518135138f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.166094 Aug 21 01:28:06.524293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 198/389] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:06 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:06.865435 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-784b5228-70df-4a44-8bd4-aec278771540 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:06.880746 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-784b5228-70df-4a44-8bd4-aec278771540 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:06.880746 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-784b5228-70df-4a44-8bd4-aec278771540 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:06.880746 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-784b5228-70df-4a44-8bd4-aec278771540 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:06.880746 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-784b5228-70df-4a44-8bd4-aec278771540 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:06.882138 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-784b5228-70df-4a44-8bd4-aec278771540 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:06.882138 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-784b5228-70df-4a44-8bd4-aec278771540 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:06.882138 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-784b5228-70df-4a44-8bd4-aec278771540 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:07.021396 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-784b5228-70df-4a44-8bd4-aec278771540 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:07.030272 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-784b5228-70df-4a44-8bd4-aec278771540 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:07.031823 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-784b5228-70df-4a44-8bd4-aec278771540 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:07.033024 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-784b5228-70df-4a44-8bd4-aec278771540 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1720 microversion: 2.20 time: 0.172402 Aug 21 01:28:07.033585 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 192/390] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:06 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1720 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:07.534827 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7f36729a-5a01-4771-b9fd-b6fc4ec0d290 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:07.536149 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7f36729a-5a01-4771-b9fd-b6fc4ec0d290 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:07.546480 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f36729a-5a01-4771-b9fd-b6fc4ec0d290 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:07.547420 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f36729a-5a01-4771-b9fd-b6fc4ec0d290 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:07.548148 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f36729a-5a01-4771-b9fd-b6fc4ec0d290 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:07.549125 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f36729a-5a01-4771-b9fd-b6fc4ec0d290 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:07.549825 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f36729a-5a01-4771-b9fd-b6fc4ec0d290 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:07.550510 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f36729a-5a01-4771-b9fd-b6fc4ec0d290 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:07.879084 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7f36729a-5a01-4771-b9fd-b6fc4ec0d290 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:07.893648 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7f36729a-5a01-4771-b9fd-b6fc4ec0d290 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.362406 Aug 21 01:28:07.895217 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 199/391] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:07 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:08.056935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d0a2594c-752e-437e-be42-a45a936ec25e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:08.057549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d0a2594c-752e-437e-be42-a45a936ec25e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:08.064000 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d0a2594c-752e-437e-be42-a45a936ec25e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:08.066778 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d0a2594c-752e-437e-be42-a45a936ec25e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:08.066778 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d0a2594c-752e-437e-be42-a45a936ec25e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:08.066778 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d0a2594c-752e-437e-be42-a45a936ec25e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:08.066778 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d0a2594c-752e-437e-be42-a45a936ec25e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:08.067132 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d0a2594c-752e-437e-be42-a45a936ec25e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:08.334782 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d0a2594c-752e-437e-be42-a45a936ec25e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:08.364676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d0a2594c-752e-437e-be42-a45a936ec25e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:08.366839 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d0a2594c-752e-437e-be42-a45a936ec25e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:08.370105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d0a2594c-752e-437e-be42-a45a936ec25e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1723 microversion: 2.20 time: 0.317296 Aug 21 01:28:08.376744 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 193/392] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:08 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1723 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:08.401714 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-584c0a45-12ab-4b56-ab33-f4e7ec223f63 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "49b53473-b9ab-4815-8822-ed1330e3657c"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:08.405204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-584c0a45-12ab-4b56-ab33-f4e7ec223f63 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:08.425962 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-584c0a45-12ab-4b56-ab33-f4e7ec223f63 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:08.426699 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-584c0a45-12ab-4b56-ab33-f4e7ec223f63 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:08.427295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-584c0a45-12ab-4b56-ab33-f4e7ec223f63 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:08.428594 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-584c0a45-12ab-4b56-ab33-f4e7ec223f63 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:08.429241 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-584c0a45-12ab-4b56-ab33-f4e7ec223f63 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:08.429845 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-584c0a45-12ab-4b56-ab33-f4e7ec223f63 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:08.806470 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-2728640f-035f-4f52-95ef-ffc640bf67f9 req-a7b3fa74-d371-4652-9263-099f554b25c0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9bc908f3-d8e3-4432-8197-51096e59a470", "tag": "9b282ab5-9500-4319-bbaa-ac3078e3128d"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:08.823474 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2728640f-035f-4f52-95ef-ffc640bf67f9 req-a7b3fa74-d371-4652-9263-099f554b25c0 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:08.830940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2728640f-035f-4f52-95ef-ffc640bf67f9 req-a7b3fa74-d371-4652-9263-099f554b25c0 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:08.833068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2728640f-035f-4f52-95ef-ffc640bf67f9 req-a7b3fa74-d371-4652-9263-099f554b25c0 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:08.853316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-2728640f-035f-4f52-95ef-ffc640bf67f9 req-a7b3fa74-d371-4652-9263-099f554b25c0 service nova] Creating event network-changed:9b282ab5-9500-4319-bbaa-ac3078e3128d for instance 9bc908f3-d8e3-4432-8197-51096e59a470 on hv-cinder-maste Aug 21 01:28:08.871202 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-2728640f-035f-4f52-95ef-ffc640bf67f9 req-a7b3fa74-d371-4652-9263-099f554b25c0 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068497 Aug 21 01:28:08.879066 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 194/393] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:28:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:28:08.925919 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bc57152f-a1d8-4abf-b9f4-987930684fa1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:08.927848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bc57152f-a1d8-4abf-b9f4-987930684fa1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:08.941902 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc57152f-a1d8-4abf-b9f4-987930684fa1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:08.946570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc57152f-a1d8-4abf-b9f4-987930684fa1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:08.946570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc57152f-a1d8-4abf-b9f4-987930684fa1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:08.946570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc57152f-a1d8-4abf-b9f4-987930684fa1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:08.946570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc57152f-a1d8-4abf-b9f4-987930684fa1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:08.947018 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc57152f-a1d8-4abf-b9f4-987930684fa1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:09.012812 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:09.146183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-bc57152f-a1d8-4abf-b9f4-987930684fa1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:09.161144 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bc57152f-a1d8-4abf-b9f4-987930684fa1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.237936 Aug 21 01:28:09.161144 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 195/394] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:08 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:09.693470 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:10.182396 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f1175b7b-e83c-48c8-a6f9-87c600978474 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:10.183104 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f1175b7b-e83c-48c8-a6f9-87c600978474 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:10.190452 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1175b7b-e83c-48c8-a6f9-87c600978474 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:10.191044 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1175b7b-e83c-48c8-a6f9-87c600978474 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:10.191406 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1175b7b-e83c-48c8-a6f9-87c600978474 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:10.192129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1175b7b-e83c-48c8-a6f9-87c600978474 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:10.192594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1175b7b-e83c-48c8-a6f9-87c600978474 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:10.193019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1175b7b-e83c-48c8-a6f9-87c600978474 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:10.329483 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-584c0a45-12ab-4b56-ab33-f4e7ec223f63 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "POST /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.935051 Aug 21 01:28:10.335494 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 200/395] 10.222.0.47 () {62 vars in 1364 bytes} [Sun Aug 21 01:28:08 2022] POST /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-volume_attachments => generated 188 bytes in 1937 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:28:10.409160 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f1175b7b-e83c-48c8-a6f9-87c600978474 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:10.418495 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f1175b7b-e83c-48c8-a6f9-87c600978474 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.241072 Aug 21 01:28:10.432321 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 196/396] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:10 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:10.523199 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-54936483-7dcd-4090-bd5d-096d3f63de9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:10.531032 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-54936483-7dcd-4090-bd5d-096d3f63de9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:10.531578 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-54936483-7dcd-4090-bd5d-096d3f63de9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:10.532363 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-54936483-7dcd-4090-bd5d-096d3f63de9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:10.533384 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-54936483-7dcd-4090-bd5d-096d3f63de9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:10.534318 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-54936483-7dcd-4090-bd5d-096d3f63de9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:10.535002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-54936483-7dcd-4090-bd5d-096d3f63de9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:10.535635 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-54936483-7dcd-4090-bd5d-096d3f63de9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:10.603922 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-54936483-7dcd-4090-bd5d-096d3f63de9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=121277) unshelve /opt/stack/nova/nova/compute/api.py:4578}} Aug 21 01:28:10.860986 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:11.023567 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-54936483-7dcd-4090-bd5d-096d3f63de9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "POST /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/action" status: 202 len: 0 microversion: 2.20 time: 0.516706 Aug 21 01:28:11.029997 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 201/397] 10.222.0.47 () {62 vars in 1319 bytes} [Sun Aug 21 01:28:10 2022] POST /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/action => generated 0 bytes in 524 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:28:11.033226 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d2a97f82-d44b-4625-a368-9db64cdd3d9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:11.033967 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d2a97f82-d44b-4625-a368-9db64cdd3d9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:11.036812 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=12 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:11.042852 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d2a97f82-d44b-4625-a368-9db64cdd3d9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:11.043360 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d2a97f82-d44b-4625-a368-9db64cdd3d9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:11.043807 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d2a97f82-d44b-4625-a368-9db64cdd3d9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:11.044659 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d2a97f82-d44b-4625-a368-9db64cdd3d9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:11.045369 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d2a97f82-d44b-4625-a368-9db64cdd3d9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:11.046012 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d2a97f82-d44b-4625-a368-9db64cdd3d9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:11.263542 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d2a97f82-d44b-4625-a368-9db64cdd3d9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:11.282057 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d2a97f82-d44b-4625-a368-9db64cdd3d9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:11.285142 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d2a97f82-d44b-4625-a368-9db64cdd3d9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:11.287235 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d2a97f82-d44b-4625-a368-9db64cdd3d9f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1749 microversion: 2.20 time: 0.258588 Aug 21 01:28:11.290454 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 197/398] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:11 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1749 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:28:11.440260 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-39678389-8951-4f82-8448-74ba9009c76a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:11.441248 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-39678389-8951-4f82-8448-74ba9009c76a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:11.448090 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-39678389-8951-4f82-8448-74ba9009c76a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:11.452794 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-39678389-8951-4f82-8448-74ba9009c76a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:11.452794 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-39678389-8951-4f82-8448-74ba9009c76a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:11.452794 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-39678389-8951-4f82-8448-74ba9009c76a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:11.452794 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-39678389-8951-4f82-8448-74ba9009c76a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:11.453247 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-39678389-8951-4f82-8448-74ba9009c76a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:11.611764 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-39678389-8951-4f82-8448-74ba9009c76a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:11.635387 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-39678389-8951-4f82-8448-74ba9009c76a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.199312 Aug 21 01:28:11.636013 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 202/399] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:11 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:12.314357 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-047ea459-0004-41de-8b18-797c67e6d7c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:12.315122 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-047ea459-0004-41de-8b18-797c67e6d7c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:12.330660 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-047ea459-0004-41de-8b18-797c67e6d7c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:12.333816 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-047ea459-0004-41de-8b18-797c67e6d7c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:12.335085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-047ea459-0004-41de-8b18-797c67e6d7c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:12.341342 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-047ea459-0004-41de-8b18-797c67e6d7c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:12.342092 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-047ea459-0004-41de-8b18-797c67e6d7c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:12.342721 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-047ea459-0004-41de-8b18-797c67e6d7c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:12.595391 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-047ea459-0004-41de-8b18-797c67e6d7c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:12.611479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-047ea459-0004-41de-8b18-797c67e6d7c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:12.613382 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-047ea459-0004-41de-8b18-797c67e6d7c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:12.617305 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-047ea459-0004-41de-8b18-797c67e6d7c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1749 microversion: 2.20 time: 0.308586 Aug 21 01:28:12.622579 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 198/400] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:12 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1749 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:12.657417 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9c2dacdb-1a39-42cf-9f93-c4c29157dcd4 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:12.658034 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9c2dacdb-1a39-42cf-9f93-c4c29157dcd4 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:12.665772 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9c2dacdb-1a39-42cf-9f93-c4c29157dcd4 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:12.672028 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9c2dacdb-1a39-42cf-9f93-c4c29157dcd4 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:12.672691 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9c2dacdb-1a39-42cf-9f93-c4c29157dcd4 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:12.676532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9c2dacdb-1a39-42cf-9f93-c4c29157dcd4 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:12.677765 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9c2dacdb-1a39-42cf-9f93-c4c29157dcd4 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:12.680145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9c2dacdb-1a39-42cf-9f93-c4c29157dcd4 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:13.029209 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9c2dacdb-1a39-42cf-9f93-c4c29157dcd4 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:13.040495 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9c2dacdb-1a39-42cf-9f93-c4c29157dcd4 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.387561 Aug 21 01:28:13.054678 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 203/401] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:12 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:13.654692 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a46a53e0-0ad8-4df0-b348-036a2fb4a454 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:13.655301 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a46a53e0-0ad8-4df0-b348-036a2fb4a454 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:13.667740 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a46a53e0-0ad8-4df0-b348-036a2fb4a454 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:13.670557 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a46a53e0-0ad8-4df0-b348-036a2fb4a454 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:13.672652 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a46a53e0-0ad8-4df0-b348-036a2fb4a454 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:13.674989 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a46a53e0-0ad8-4df0-b348-036a2fb4a454 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:13.677326 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a46a53e0-0ad8-4df0-b348-036a2fb4a454 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:13.678019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a46a53e0-0ad8-4df0-b348-036a2fb4a454 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:14.066678 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9ef6ba5b-0320-4fc1-9d49-6da4acb00a06 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:14.067310 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9ef6ba5b-0320-4fc1-9d49-6da4acb00a06 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:14.090951 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ef6ba5b-0320-4fc1-9d49-6da4acb00a06 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:14.095270 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ef6ba5b-0320-4fc1-9d49-6da4acb00a06 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:14.095721 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ef6ba5b-0320-4fc1-9d49-6da4acb00a06 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:14.097496 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ef6ba5b-0320-4fc1-9d49-6da4acb00a06 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:14.097846 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ef6ba5b-0320-4fc1-9d49-6da4acb00a06 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:14.100780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ef6ba5b-0320-4fc1-9d49-6da4acb00a06 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:14.342987 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a46a53e0-0ad8-4df0-b348-036a2fb4a454 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:14.367304 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a46a53e0-0ad8-4df0-b348-036a2fb4a454 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:14.380602 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a46a53e0-0ad8-4df0-b348-036a2fb4a454 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:14.387807 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a46a53e0-0ad8-4df0-b348-036a2fb4a454 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1749 microversion: 2.20 time: 0.737746 Aug 21 01:28:14.388503 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 199/402] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:13 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1749 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:14.718795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9ef6ba5b-0320-4fc1-9d49-6da4acb00a06 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:14.740218 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9ef6ba5b-0320-4fc1-9d49-6da4acb00a06 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.678780 Aug 21 01:28:14.740925 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 204/403] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:14 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 680 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:15.418608 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e3caea0b-a5c9-4953-b1ca-9e33f03f948e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:15.419998 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e3caea0b-a5c9-4953-b1ca-9e33f03f948e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:15.440267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e3caea0b-a5c9-4953-b1ca-9e33f03f948e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:15.440779 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e3caea0b-a5c9-4953-b1ca-9e33f03f948e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:15.441223 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e3caea0b-a5c9-4953-b1ca-9e33f03f948e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:15.441940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e3caea0b-a5c9-4953-b1ca-9e33f03f948e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:15.442709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e3caea0b-a5c9-4953-b1ca-9e33f03f948e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:15.445882 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e3caea0b-a5c9-4953-b1ca-9e33f03f948e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:15.777386 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-fdb60f65-c091-4f57-a145-62b5be1b694e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:15.778037 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-fdb60f65-c091-4f57-a145-62b5be1b694e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:15.801957 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fdb60f65-c091-4f57-a145-62b5be1b694e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:15.802364 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fdb60f65-c091-4f57-a145-62b5be1b694e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:15.802815 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fdb60f65-c091-4f57-a145-62b5be1b694e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:15.803525 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fdb60f65-c091-4f57-a145-62b5be1b694e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:15.807458 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fdb60f65-c091-4f57-a145-62b5be1b694e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:15.811178 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fdb60f65-c091-4f57-a145-62b5be1b694e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:15.890063 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e3caea0b-a5c9-4953-b1ca-9e33f03f948e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:15.960048 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e3caea0b-a5c9-4953-b1ca-9e33f03f948e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:15.960048 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e3caea0b-a5c9-4953-b1ca-9e33f03f948e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:15.960048 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e3caea0b-a5c9-4953-b1ca-9e33f03f948e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1751 microversion: 2.20 time: 0.503015 Aug 21 01:28:15.961588 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 200/404] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:15 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1751 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:16.053058 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-fdb60f65-c091-4f57-a145-62b5be1b694e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:16.063590 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-fdb60f65-c091-4f57-a145-62b5be1b694e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.293006 Aug 21 01:28:16.091392 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 205/405] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:15 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:16.951020 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f6f66acf-626e-460e-96a4-df41e8a13bd2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:16.968301 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f6f66acf-626e-460e-96a4-df41e8a13bd2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:16.968301 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f6f66acf-626e-460e-96a4-df41e8a13bd2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:16.968301 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f6f66acf-626e-460e-96a4-df41e8a13bd2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:16.968301 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f6f66acf-626e-460e-96a4-df41e8a13bd2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:16.969119 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f6f66acf-626e-460e-96a4-df41e8a13bd2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:16.969119 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f6f66acf-626e-460e-96a4-df41e8a13bd2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:16.969119 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f6f66acf-626e-460e-96a4-df41e8a13bd2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:17.089750 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a0a29b8e-2103-49a1-8c4a-9944ab73aa6f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:17.091179 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a0a29b8e-2103-49a1-8c4a-9944ab73aa6f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:17.094069 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=85,nova_api:UPDATE=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:17.108318 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a0a29b8e-2103-49a1-8c4a-9944ab73aa6f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:17.112589 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a0a29b8e-2103-49a1-8c4a-9944ab73aa6f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:17.112589 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a0a29b8e-2103-49a1-8c4a-9944ab73aa6f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:17.120970 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a0a29b8e-2103-49a1-8c4a-9944ab73aa6f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:17.121868 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a0a29b8e-2103-49a1-8c4a-9944ab73aa6f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:17.122515 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a0a29b8e-2103-49a1-8c4a-9944ab73aa6f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:17.375448 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f6f66acf-626e-460e-96a4-df41e8a13bd2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:17.411297 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f6f66acf-626e-460e-96a4-df41e8a13bd2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:17.421497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f6f66acf-626e-460e-96a4-df41e8a13bd2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:17.426177 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f6f66acf-626e-460e-96a4-df41e8a13bd2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1751 microversion: 2.20 time: 0.485473 Aug 21 01:28:17.442234 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 201/406] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:16 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1751 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:17.532835 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a0a29b8e-2103-49a1-8c4a-9944ab73aa6f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:17.541456 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a0a29b8e-2103-49a1-8c4a-9944ab73aa6f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.455476 Aug 21 01:28:17.542381 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 206/407] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:17 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:18.456912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3adb4b6d-48d4-426a-b978-f239e1b08864 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:18.457831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3adb4b6d-48d4-426a-b978-f239e1b08864 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:18.468867 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3adb4b6d-48d4-426a-b978-f239e1b08864 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:18.473432 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3adb4b6d-48d4-426a-b978-f239e1b08864 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:18.474339 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3adb4b6d-48d4-426a-b978-f239e1b08864 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:18.475119 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3adb4b6d-48d4-426a-b978-f239e1b08864 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:18.475620 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3adb4b6d-48d4-426a-b978-f239e1b08864 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:18.476028 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3adb4b6d-48d4-426a-b978-f239e1b08864 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:18.563109 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-37c57920-2af3-48b3-9b65-947ba5d46039 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:18.564551 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-37c57920-2af3-48b3-9b65-947ba5d46039 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:18.577595 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37c57920-2af3-48b3-9b65-947ba5d46039 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:18.581120 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37c57920-2af3-48b3-9b65-947ba5d46039 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:18.581822 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37c57920-2af3-48b3-9b65-947ba5d46039 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:18.582755 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37c57920-2af3-48b3-9b65-947ba5d46039 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:18.583384 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37c57920-2af3-48b3-9b65-947ba5d46039 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:18.586116 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37c57920-2af3-48b3-9b65-947ba5d46039 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:18.956158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3adb4b6d-48d4-426a-b978-f239e1b08864 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:18.969082 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3adb4b6d-48d4-426a-b978-f239e1b08864 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:18.969310 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3adb4b6d-48d4-426a-b978-f239e1b08864 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:18.971535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3adb4b6d-48d4-426a-b978-f239e1b08864 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1751 microversion: 2.20 time: 0.519512 Aug 21 01:28:18.972748 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 202/408] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:18 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1751 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:18.995486 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-37c57920-2af3-48b3-9b65-947ba5d46039 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:19.004221 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-37c57920-2af3-48b3-9b65-947ba5d46039 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.444861 Aug 21 01:28:19.004799 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 207/409] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:18 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:20.002322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-32494f2e-0d95-42f3-a26c-92aad6eb6a03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:20.002994 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-32494f2e-0d95-42f3-a26c-92aad6eb6a03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:20.010097 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-32494f2e-0d95-42f3-a26c-92aad6eb6a03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:20.010599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-32494f2e-0d95-42f3-a26c-92aad6eb6a03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:20.011043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-32494f2e-0d95-42f3-a26c-92aad6eb6a03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:20.011904 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-32494f2e-0d95-42f3-a26c-92aad6eb6a03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:20.012404 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-32494f2e-0d95-42f3-a26c-92aad6eb6a03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:20.012869 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-32494f2e-0d95-42f3-a26c-92aad6eb6a03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:20.029796 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-21c715ba-825c-4ce2-bf52-890b495db0f9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:20.030274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-21c715ba-825c-4ce2-bf52-890b495db0f9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:20.042852 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-21c715ba-825c-4ce2-bf52-890b495db0f9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:20.043462 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-21c715ba-825c-4ce2-bf52-890b495db0f9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:20.049263 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-21c715ba-825c-4ce2-bf52-890b495db0f9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:20.057098 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-21c715ba-825c-4ce2-bf52-890b495db0f9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:20.057098 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-21c715ba-825c-4ce2-bf52-890b495db0f9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:20.057098 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-21c715ba-825c-4ce2-bf52-890b495db0f9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:20.308418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-32494f2e-0d95-42f3-a26c-92aad6eb6a03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:20.327055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-32494f2e-0d95-42f3-a26c-92aad6eb6a03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:20.333983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-32494f2e-0d95-42f3-a26c-92aad6eb6a03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:20.338907 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-32494f2e-0d95-42f3-a26c-92aad6eb6a03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1751 microversion: 2.20 time: 0.344045 Aug 21 01:28:20.350474 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 203/410] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:19 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1751 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:20.401260 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-21c715ba-825c-4ce2-bf52-890b495db0f9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:20.413702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-21c715ba-825c-4ce2-bf52-890b495db0f9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.387781 Aug 21 01:28:20.414913 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 208/411] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:20 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:21.385119 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e2c41f68-1653-4039-83c9-e785df24b5dd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:21.385815 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e2c41f68-1653-4039-83c9-e785df24b5dd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:21.393280 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e2c41f68-1653-4039-83c9-e785df24b5dd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:21.393763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e2c41f68-1653-4039-83c9-e785df24b5dd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:21.394196 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e2c41f68-1653-4039-83c9-e785df24b5dd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:21.394935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e2c41f68-1653-4039-83c9-e785df24b5dd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:21.395384 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e2c41f68-1653-4039-83c9-e785df24b5dd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:21.395811 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e2c41f68-1653-4039-83c9-e785df24b5dd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:21.399952 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=100 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:21.443964 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7969cfd0-d2b4-4d6e-bb86-83fc8b043dca tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:21.444813 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7969cfd0-d2b4-4d6e-bb86-83fc8b043dca tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:21.451994 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7969cfd0-d2b4-4d6e-bb86-83fc8b043dca tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:21.452726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7969cfd0-d2b4-4d6e-bb86-83fc8b043dca tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:21.453345 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7969cfd0-d2b4-4d6e-bb86-83fc8b043dca tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:21.454235 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7969cfd0-d2b4-4d6e-bb86-83fc8b043dca tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:21.454698 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7969cfd0-d2b4-4d6e-bb86-83fc8b043dca tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:21.455146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7969cfd0-d2b4-4d6e-bb86-83fc8b043dca tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:21.611624 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e2c41f68-1653-4039-83c9-e785df24b5dd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:21.622805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e2c41f68-1653-4039-83c9-e785df24b5dd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:21.624361 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e2c41f68-1653-4039-83c9-e785df24b5dd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:21.625763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e2c41f68-1653-4039-83c9-e785df24b5dd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1751 microversion: 2.20 time: 0.244453 Aug 21 01:28:21.626574 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 204/412] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:21 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1751 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:21.673754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7969cfd0-d2b4-4d6e-bb86-83fc8b043dca tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:21.683075 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7969cfd0-d2b4-4d6e-bb86-83fc8b043dca tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.249753 Aug 21 01:28:21.683918 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 209/413] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:21 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:22.649527 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6bf6408e-3b3e-4cf6-83e2-286dc2498c1e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:22.655197 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6bf6408e-3b3e-4cf6-83e2-286dc2498c1e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:22.658996 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6bf6408e-3b3e-4cf6-83e2-286dc2498c1e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:22.660397 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6bf6408e-3b3e-4cf6-83e2-286dc2498c1e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:22.661105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6bf6408e-3b3e-4cf6-83e2-286dc2498c1e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:22.662892 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6bf6408e-3b3e-4cf6-83e2-286dc2498c1e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:22.664398 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6bf6408e-3b3e-4cf6-83e2-286dc2498c1e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:22.665121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6bf6408e-3b3e-4cf6-83e2-286dc2498c1e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:22.703991 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3bfea539-e243-4985-80da-23189271f4e2 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:22.705371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3bfea539-e243-4985-80da-23189271f4e2 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:22.720391 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3bfea539-e243-4985-80da-23189271f4e2 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:22.723795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3bfea539-e243-4985-80da-23189271f4e2 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:22.724578 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3bfea539-e243-4985-80da-23189271f4e2 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:22.725561 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3bfea539-e243-4985-80da-23189271f4e2 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:22.727208 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3bfea539-e243-4985-80da-23189271f4e2 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:22.727895 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3bfea539-e243-4985-80da-23189271f4e2 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:23.337397 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6bf6408e-3b3e-4cf6-83e2-286dc2498c1e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:23.376757 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6bf6408e-3b3e-4cf6-83e2-286dc2498c1e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:23.380166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6bf6408e-3b3e-4cf6-83e2-286dc2498c1e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:23.382518 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6bf6408e-3b3e-4cf6-83e2-286dc2498c1e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.737307 Aug 21 01:28:23.383547 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 205/414] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:22 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 739 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:23.385760 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3bfea539-e243-4985-80da-23189271f4e2 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:23.410771 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3bfea539-e243-4985-80da-23189271f4e2 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.710789 Aug 21 01:28:23.411369 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 210/415] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:22 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:24.416218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d5a960ac-67ee-49f3-a018-f5317f73fe9d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:24.416863 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d5a960ac-67ee-49f3-a018-f5317f73fe9d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:24.423577 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5a960ac-67ee-49f3-a018-f5317f73fe9d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:24.424224 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5a960ac-67ee-49f3-a018-f5317f73fe9d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:24.424667 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5a960ac-67ee-49f3-a018-f5317f73fe9d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:24.425367 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5a960ac-67ee-49f3-a018-f5317f73fe9d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:24.425823 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5a960ac-67ee-49f3-a018-f5317f73fe9d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:24.426280 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5a960ac-67ee-49f3-a018-f5317f73fe9d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:24.434023 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3225432a-63e8-4629-880b-5d3a71350b4e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:24.434636 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3225432a-63e8-4629-880b-5d3a71350b4e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:24.441753 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3225432a-63e8-4629-880b-5d3a71350b4e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:24.444161 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3225432a-63e8-4629-880b-5d3a71350b4e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:24.444161 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3225432a-63e8-4629-880b-5d3a71350b4e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:24.444415 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3225432a-63e8-4629-880b-5d3a71350b4e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:24.444884 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3225432a-63e8-4629-880b-5d3a71350b4e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:24.445735 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3225432a-63e8-4629-880b-5d3a71350b4e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:25.265398 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d5a960ac-67ee-49f3-a018-f5317f73fe9d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:25.279838 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d5a960ac-67ee-49f3-a018-f5317f73fe9d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:25.283941 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d5a960ac-67ee-49f3-a018-f5317f73fe9d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:25.284156 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3225432a-63e8-4629-880b-5d3a71350b4e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:25.289895 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d5a960ac-67ee-49f3-a018-f5317f73fe9d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.877759 Aug 21 01:28:25.290735 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 206/416] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:24 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 879 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:25.303699 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3225432a-63e8-4629-880b-5d3a71350b4e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.874359 Aug 21 01:28:25.304713 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 211/417] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:24 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 876 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:26.314546 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fe849bfd-8517-416e-afc3-4afbf79862ba tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:26.315129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fe849bfd-8517-416e-afc3-4afbf79862ba tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:26.335461 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6b6544c6-4f97-4473-8ce3-f2fc9c3beaff tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:26.336378 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6b6544c6-4f97-4473-8ce3-f2fc9c3beaff tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:26.343164 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe849bfd-8517-416e-afc3-4afbf79862ba tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:26.343164 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe849bfd-8517-416e-afc3-4afbf79862ba tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:26.343164 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe849bfd-8517-416e-afc3-4afbf79862ba tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:26.343164 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe849bfd-8517-416e-afc3-4afbf79862ba tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:26.343848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe849bfd-8517-416e-afc3-4afbf79862ba tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:26.343848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe849bfd-8517-416e-afc3-4afbf79862ba tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:26.360168 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6b6544c6-4f97-4473-8ce3-f2fc9c3beaff tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:26.361646 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6b6544c6-4f97-4473-8ce3-f2fc9c3beaff tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:26.364132 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6b6544c6-4f97-4473-8ce3-f2fc9c3beaff tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:26.365095 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6b6544c6-4f97-4473-8ce3-f2fc9c3beaff tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:26.366587 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6b6544c6-4f97-4473-8ce3-f2fc9c3beaff tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:26.367723 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6b6544c6-4f97-4473-8ce3-f2fc9c3beaff tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:26.933687 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6b6544c6-4f97-4473-8ce3-f2fc9c3beaff tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:26.956518 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6b6544c6-4f97-4473-8ce3-f2fc9c3beaff tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.630597 Aug 21 01:28:26.959900 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 212/418] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:26 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:26.975097 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fe849bfd-8517-416e-afc3-4afbf79862ba tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:26.989853 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fe849bfd-8517-416e-afc3-4afbf79862ba tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:26.993045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fe849bfd-8517-416e-afc3-4afbf79862ba tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:26.998837 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fe849bfd-8517-416e-afc3-4afbf79862ba tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.688173 Aug 21 01:28:27.001806 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 207/419] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:26 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 692 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:27.374144 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-19af7837-8aea-457b-bb74-3f0d55a3aa01 req-a526528c-9c9a-49d3-9fd8-19210ed1fc57 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9bc908f3-d8e3-4432-8197-51096e59a470", "name": "network-vif-plugged", "status": "completed", "tag": "9b282ab5-9500-4319-bbaa-ac3078e3128d"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:27.387057 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-19af7837-8aea-457b-bb74-3f0d55a3aa01 req-a526528c-9c9a-49d3-9fd8-19210ed1fc57 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:27.387922 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-19af7837-8aea-457b-bb74-3f0d55a3aa01 req-a526528c-9c9a-49d3-9fd8-19210ed1fc57 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:27.388642 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-19af7837-8aea-457b-bb74-3f0d55a3aa01 req-a526528c-9c9a-49d3-9fd8-19210ed1fc57 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:27.403275 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-19af7837-8aea-457b-bb74-3f0d55a3aa01 req-a526528c-9c9a-49d3-9fd8-19210ed1fc57 service nova] Creating event network-vif-plugged:9b282ab5-9500-4319-bbaa-ac3078e3128d for instance 9bc908f3-d8e3-4432-8197-51096e59a470 on hv-cinder-maste Aug 21 01:28:27.412059 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-19af7837-8aea-457b-bb74-3f0d55a3aa01 req-a526528c-9c9a-49d3-9fd8-19210ed1fc57 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043122 Aug 21 01:28:27.414439 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 213/420] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:28:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:28:27.985844 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-dd851790-37c4-4120-999b-3c535461ec07 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:27.986804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-dd851790-37c4-4120-999b-3c535461ec07 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:27.994168 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dd851790-37c4-4120-999b-3c535461ec07 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:27.994708 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dd851790-37c4-4120-999b-3c535461ec07 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:27.995184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dd851790-37c4-4120-999b-3c535461ec07 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:27.995894 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dd851790-37c4-4120-999b-3c535461ec07 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:27.996390 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dd851790-37c4-4120-999b-3c535461ec07 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:27.996872 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dd851790-37c4-4120-999b-3c535461ec07 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:28.030956 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-daf2c44e-4c39-4e8a-ae94-1170808d80af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:28.031663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-daf2c44e-4c39-4e8a-ae94-1170808d80af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:28.043269 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-daf2c44e-4c39-4e8a-ae94-1170808d80af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:28.043771 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-daf2c44e-4c39-4e8a-ae94-1170808d80af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:28.044240 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-daf2c44e-4c39-4e8a-ae94-1170808d80af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:28.048444 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-daf2c44e-4c39-4e8a-ae94-1170808d80af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:28.049112 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-daf2c44e-4c39-4e8a-ae94-1170808d80af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:28.049706 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-daf2c44e-4c39-4e8a-ae94-1170808d80af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:28.340337 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-daf2c44e-4c39-4e8a-ae94-1170808d80af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:28.352126 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-dd851790-37c4-4120-999b-3c535461ec07 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:28.356234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-daf2c44e-4c39-4e8a-ae94-1170808d80af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:28.360119 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-daf2c44e-4c39-4e8a-ae94-1170808d80af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:28.361688 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-daf2c44e-4c39-4e8a-ae94-1170808d80af tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.335580 Aug 21 01:28:28.363051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 214/421] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:28 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:28:28.365368 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-dd851790-37c4-4120-999b-3c535461ec07 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.383113 Aug 21 01:28:28.365802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 208/422] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:27 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:29.406934 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2a731e6d-fbda-43b7-ab8c-ab24197f44b9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:29.410391 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-2a731e6d-fbda-43b7-ab8c-ab24197f44b9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:29.410540 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7574c0dd-0fc5-45ad-a39a-bf5031a55e24 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:29.411241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7574c0dd-0fc5-45ad-a39a-bf5031a55e24 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:29.422346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7574c0dd-0fc5-45ad-a39a-bf5031a55e24 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:29.423660 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7574c0dd-0fc5-45ad-a39a-bf5031a55e24 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:29.424542 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7574c0dd-0fc5-45ad-a39a-bf5031a55e24 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:29.425722 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7574c0dd-0fc5-45ad-a39a-bf5031a55e24 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:29.426346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7574c0dd-0fc5-45ad-a39a-bf5031a55e24 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:29.433269 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2a731e6d-fbda-43b7-ab8c-ab24197f44b9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:29.435858 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7574c0dd-0fc5-45ad-a39a-bf5031a55e24 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:29.439389 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2a731e6d-fbda-43b7-ab8c-ab24197f44b9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:29.440020 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2a731e6d-fbda-43b7-ab8c-ab24197f44b9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:29.440987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2a731e6d-fbda-43b7-ab8c-ab24197f44b9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:29.441444 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2a731e6d-fbda-43b7-ab8c-ab24197f44b9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:29.445498 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2a731e6d-fbda-43b7-ab8c-ab24197f44b9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:29.951264 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7574c0dd-0fc5-45ad-a39a-bf5031a55e24 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:29.968025 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7574c0dd-0fc5-45ad-a39a-bf5031a55e24 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:29.970027 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7574c0dd-0fc5-45ad-a39a-bf5031a55e24 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:29.971509 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7574c0dd-0fc5-45ad-a39a-bf5031a55e24 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.566895 Aug 21 01:28:29.988948 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 209/423] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:29 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:30.027897 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2a731e6d-fbda-43b7-ab8c-ab24197f44b9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:30.038456 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2a731e6d-fbda-43b7-ab8c-ab24197f44b9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.639375 Aug 21 01:28:30.039275 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 215/424] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:29 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 641 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:30.898312 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bb42a090-8077-45fa-b24b-2a8dbb9074fc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Action: 'create', calling method: .version_select at 0x7f9da34cb5e0>, body: {"keypair": {"name": "tempest-TestShelveInstance-281379194", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLBSqoRof7rmTHEc3gWrIHRPx2Fq2wnSRP2KUGK86boaAhTnxAltk/OjfsuO/gzQ2qlsYn9TfknA6dXJ0Z1hDHrvFwwVJerGtRjdNh5fV3XuFQlNUwswwZ5WNhKpvIzcDA=="}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:30.915460 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-bb42a090-8077-45fa-b24b-2a8dbb9074fc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Getting quotas for project ac06ecc6fc93477ba4909284525df514. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:28:30.919920 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-bb42a090-8077-45fa-b24b-2a8dbb9074fc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Getting quotas for user 03d34d1e66404d259599806d040628ef and project ac06ecc6fc93477ba4909284525df514. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:28:30.997411 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4b8dca94-059f-4ba3-92c1-5a15ed578455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:31.000602 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4b8dca94-059f-4ba3-92c1-5a15ed578455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:31.011431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4b8dca94-059f-4ba3-92c1-5a15ed578455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:31.011952 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4b8dca94-059f-4ba3-92c1-5a15ed578455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:31.016734 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4b8dca94-059f-4ba3-92c1-5a15ed578455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:31.017908 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4b8dca94-059f-4ba3-92c1-5a15ed578455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:31.018395 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4b8dca94-059f-4ba3-92c1-5a15ed578455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:31.018834 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4b8dca94-059f-4ba3-92c1-5a15ed578455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:31.022841 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bb42a090-8077-45fa-b24b-2a8dbb9074fc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.612505 Aug 21 01:28:31.024547 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 210/425] 10.222.0.47 () {60 vars in 1159 bytes} [Sun Aug 21 01:28:30 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:28:31.070592 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9b854bf7-2635-4ac8-8a09-d60d5753eb4c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:31.073166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9b854bf7-2635-4ac8-8a09-d60d5753eb4c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:31.087789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9b854bf7-2635-4ac8-8a09-d60d5753eb4c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:31.088399 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9b854bf7-2635-4ac8-8a09-d60d5753eb4c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:31.088926 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9b854bf7-2635-4ac8-8a09-d60d5753eb4c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:31.089717 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9b854bf7-2635-4ac8-8a09-d60d5753eb4c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:31.090329 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9b854bf7-2635-4ac8-8a09-d60d5753eb4c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:31.090845 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9b854bf7-2635-4ac8-8a09-d60d5753eb4c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:31.352203 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9b854bf7-2635-4ac8-8a09-d60d5753eb4c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:31.366657 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9b854bf7-2635-4ac8-8a09-d60d5753eb4c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1357 microversion: 2.1 time: 0.311437 Aug 21 01:28:31.367728 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 211/426] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:31 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1357 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:31.473889 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4b8dca94-059f-4ba3-92c1-5a15ed578455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:31.502175 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4b8dca94-059f-4ba3-92c1-5a15ed578455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:31.506106 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4b8dca94-059f-4ba3-92c1-5a15ed578455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:31.520428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4b8dca94-059f-4ba3-92c1-5a15ed578455 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.515293 Aug 21 01:28:31.520428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 216/427] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:30 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:32.389103 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4c9f7bd6-a34f-412f-9095-bb4f6f7546e3 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:32.390466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4c9f7bd6-a34f-412f-9095-bb4f6f7546e3 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:32.393329 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=82,nova_api:INSERT=4 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:32.407285 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4c9f7bd6-a34f-412f-9095-bb4f6f7546e3 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:32.407285 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4c9f7bd6-a34f-412f-9095-bb4f6f7546e3 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:32.407285 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4c9f7bd6-a34f-412f-9095-bb4f6f7546e3 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:32.413866 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4c9f7bd6-a34f-412f-9095-bb4f6f7546e3 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:32.413866 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4c9f7bd6-a34f-412f-9095-bb4f6f7546e3 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:32.415998 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4c9f7bd6-a34f-412f-9095-bb4f6f7546e3 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:32.536953 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-98df431a-8290-4497-b99a-3246842aa7e5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:32.537632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-98df431a-8290-4497-b99a-3246842aa7e5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:32.551291 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-98df431a-8290-4497-b99a-3246842aa7e5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:32.557378 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-98df431a-8290-4497-b99a-3246842aa7e5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:32.557378 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-98df431a-8290-4497-b99a-3246842aa7e5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:32.557378 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-98df431a-8290-4497-b99a-3246842aa7e5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:32.557378 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-98df431a-8290-4497-b99a-3246842aa7e5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:32.557833 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-98df431a-8290-4497-b99a-3246842aa7e5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:32.681964 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4c9f7bd6-a34f-412f-9095-bb4f6f7546e3 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:32.696980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4c9f7bd6-a34f-412f-9095-bb4f6f7546e3 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1548 microversion: 2.1 time: 0.310013 Aug 21 01:28:32.700009 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 212/428] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:32 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1548 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:32.737731 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ae0a3fb8-77d5-4cc8-a7b3-ef207faedf68 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:32.738375 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ae0a3fb8-77d5-4cc8-a7b3-ef207faedf68 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:32.746287 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae0a3fb8-77d5-4cc8-a7b3-ef207faedf68 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:32.748364 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae0a3fb8-77d5-4cc8-a7b3-ef207faedf68 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:32.749000 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae0a3fb8-77d5-4cc8-a7b3-ef207faedf68 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:32.752093 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae0a3fb8-77d5-4cc8-a7b3-ef207faedf68 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:32.753015 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae0a3fb8-77d5-4cc8-a7b3-ef207faedf68 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:32.754186 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae0a3fb8-77d5-4cc8-a7b3-ef207faedf68 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:32.839085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-98df431a-8290-4497-b99a-3246842aa7e5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:32.856594 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-98df431a-8290-4497-b99a-3246842aa7e5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:32.858149 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-98df431a-8290-4497-b99a-3246842aa7e5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:32.863161 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-98df431a-8290-4497-b99a-3246842aa7e5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.331203 Aug 21 01:28:32.864749 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 217/429] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:32 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:28:33.096602 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ae0a3fb8-77d5-4cc8-a7b3-ef207faedf68 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:33.106354 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ae0a3fb8-77d5-4cc8-a7b3-ef207faedf68 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1548 microversion: 2.1 time: 0.385625 Aug 21 01:28:33.112551 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 213/430] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:32 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1548 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:33.137260 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f6de8240-0b72-49cc-8ba1-9956a3edc8db tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:33.158823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f6de8240-0b72-49cc-8ba1-9956a3edc8db tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:33.160943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f6de8240-0b72-49cc-8ba1-9956a3edc8db tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Searching by: {'deleted': False, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149'} {{(pid=121277) get_all /opt/stack/nova/nova/compute/api.py:3004}} Aug 21 01:28:33.173491 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f6de8240-0b72-49cc-8ba1-9956a3edc8db tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:33.174205 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f6de8240-0b72-49cc-8ba1-9956a3edc8db tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:33.175361 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f6de8240-0b72-49cc-8ba1-9956a3edc8db tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:33.176822 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f6de8240-0b72-49cc-8ba1-9956a3edc8db tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:33.177815 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f6de8240-0b72-49cc-8ba1-9956a3edc8db tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:33.179299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f6de8240-0b72-49cc-8ba1-9956a3edc8db tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:33.392790 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.multi_cell_list [None req-f6de8240-0b72-49cc-8ba1-9956a3edc8db tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=121277) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Aug 21 01:28:33.479710 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=100 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:33.498712 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f6de8240-0b72-49cc-8ba1-9956a3edc8db tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers" status: 200 len: 341 microversion: 2.1 time: 0.369227 Aug 21 01:28:33.510950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e637ebf3-a8aa-42ac-a688-979641a63e82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:33.512275 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e637ebf3-a8aa-42ac-a688-979641a63e82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:33.518803 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 218/431] 10.222.0.47 () {58 vars in 1125 bytes} [Sun Aug 21 01:28:33 2022] GET /compute/v2.1/servers => generated 341 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:28:33.522320 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e637ebf3-a8aa-42ac-a688-979641a63e82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:33.525882 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e637ebf3-a8aa-42ac-a688-979641a63e82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:33.525882 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e637ebf3-a8aa-42ac-a688-979641a63e82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:33.525882 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e637ebf3-a8aa-42ac-a688-979641a63e82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:33.525882 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e637ebf3-a8aa-42ac-a688-979641a63e82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:33.526236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e637ebf3-a8aa-42ac-a688-979641a63e82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:33.839894 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e637ebf3-a8aa-42ac-a688-979641a63e82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:33.859964 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e637ebf3-a8aa-42ac-a688-979641a63e82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1548 microversion: 2.1 time: 0.351832 Aug 21 01:28:33.861488 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 214/432] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:33 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1548 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:33.906205 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b10c8dfe-fd4e-4538-8f7e-959f27368435 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:33.907431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b10c8dfe-fd4e-4538-8f7e-959f27368435 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:33.916032 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b10c8dfe-fd4e-4538-8f7e-959f27368435 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:33.916737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b10c8dfe-fd4e-4538-8f7e-959f27368435 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:33.917424 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b10c8dfe-fd4e-4538-8f7e-959f27368435 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:33.918803 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b10c8dfe-fd4e-4538-8f7e-959f27368435 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:33.919435 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b10c8dfe-fd4e-4538-8f7e-959f27368435 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:33.920130 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b10c8dfe-fd4e-4538-8f7e-959f27368435 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:34.878517 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b10c8dfe-fd4e-4538-8f7e-959f27368435 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:34.889599 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b10c8dfe-fd4e-4538-8f7e-959f27368435 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:34.899285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b10c8dfe-fd4e-4538-8f7e-959f27368435 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:34.900881 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b10c8dfe-fd4e-4538-8f7e-959f27368435 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 1.003537 Aug 21 01:28:34.916256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 219/433] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:33 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 1007 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:35.926871 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-28994a0d-9b16-4954-915c-460a5a8ec5a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:35.927737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-28994a0d-9b16-4954-915c-460a5a8ec5a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:35.938420 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28994a0d-9b16-4954-915c-460a5a8ec5a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:35.942497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28994a0d-9b16-4954-915c-460a5a8ec5a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:35.942497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28994a0d-9b16-4954-915c-460a5a8ec5a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:35.942497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28994a0d-9b16-4954-915c-460a5a8ec5a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:35.942497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28994a0d-9b16-4954-915c-460a5a8ec5a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:35.943353 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28994a0d-9b16-4954-915c-460a5a8ec5a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:36.141896 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-28994a0d-9b16-4954-915c-460a5a8ec5a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:36.152288 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-28994a0d-9b16-4954-915c-460a5a8ec5a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:36.154087 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-28994a0d-9b16-4954-915c-460a5a8ec5a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:36.155441 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-28994a0d-9b16-4954-915c-460a5a8ec5a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.232787 Aug 21 01:28:36.156559 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 215/434] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:35 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:36.269562 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-30e22a70-e61f-42f5-ba9a-636506009f1b tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "15bddfef-2aaf-400f-80f6-4ea3ebd9c646"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:36.272038 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-30e22a70-e61f-42f5-ba9a-636506009f1b tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:36.279394 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-30e22a70-e61f-42f5-ba9a-636506009f1b tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:36.279877 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-30e22a70-e61f-42f5-ba9a-636506009f1b tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:36.280347 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-30e22a70-e61f-42f5-ba9a-636506009f1b tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:36.281055 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-30e22a70-e61f-42f5-ba9a-636506009f1b tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:36.281508 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-30e22a70-e61f-42f5-ba9a-636506009f1b tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:36.281930 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-30e22a70-e61f-42f5-ba9a-636506009f1b tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:36.593533 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1528798872", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "networks": [{"uuid": "5a72733d-0e4b-47b7-9270-d8de517ee151"}]}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:36.650921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] validate_networks() for [('5a72733d-0e4b-47b7-9270-d8de517ee151', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:28:37.083485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:28:37.084059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:28:37.084400 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:28:37.084821 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:28:37.085234 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:28:37.098034 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:37.098540 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:37.098960 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:37.099688 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:37.100181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:37.100636 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:37.120419 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Getting quotas for project 67f492f96d314cfea70bdbfa3caaee9e. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:28:37.127719 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Getting quotas for user 2325a3d547d043d2a9ac95986187027b and project 67f492f96d314cfea70bdbfa3caaee9e. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:28:37.141079 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:28:37.143389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:28:37.281680 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9c87ba87-847b-4e82-8902-7359f7052f06 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.691919 Aug 21 01:28:37.282535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 216/435] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:28:36 2022] POST /compute/v2.1/servers => generated 380 bytes in 693 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 21 01:28:37.286858 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-0bb6494d-d947-47c8-9ec6-26a96e1be43b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:37.287469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-0bb6494d-d947-47c8-9ec6-26a96e1be43b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:37.293802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0bb6494d-d947-47c8-9ec6-26a96e1be43b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:37.294298 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0bb6494d-d947-47c8-9ec6-26a96e1be43b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:37.294749 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0bb6494d-d947-47c8-9ec6-26a96e1be43b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:37.295583 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0bb6494d-d947-47c8-9ec6-26a96e1be43b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:37.296043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0bb6494d-d947-47c8-9ec6-26a96e1be43b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:37.296506 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0bb6494d-d947-47c8-9ec6-26a96e1be43b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:37.355670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-30e22a70-e61f-42f5-ba9a-636506009f1b tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "POST /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.091984 Aug 21 01:28:37.366818 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 220/436] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:28:36 2022] POST /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments => generated 194 bytes in 1106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:28:37.376684 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e560d9fc-41d1-4cde-a969-9fdd47c4e19a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:37.379506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e560d9fc-41d1-4cde-a969-9fdd47c4e19a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:37.476061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e560d9fc-41d1-4cde-a969-9fdd47c4e19a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:37.501059 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e560d9fc-41d1-4cde-a969-9fdd47c4e19a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1219 microversion: 2.1 time: 0.132476 Aug 21 01:28:37.501795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 221/437] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:37 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1219 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:37.558145 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0bb6494d-d947-47c8-9ec6-26a96e1be43b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:37.569349 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0bb6494d-d947-47c8-9ec6-26a96e1be43b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:37.571719 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0bb6494d-d947-47c8-9ec6-26a96e1be43b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:37.572964 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-0bb6494d-d947-47c8-9ec6-26a96e1be43b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.289684 Aug 21 01:28:37.573707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 217/438] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:37 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:38.524268 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-db04f81c-8d3f-43d9-835a-5e56078b6783 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:38.525160 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-db04f81c-8d3f-43d9-835a-5e56078b6783 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:38.536393 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-db04f81c-8d3f-43d9-835a-5e56078b6783 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:38.537217 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-db04f81c-8d3f-43d9-835a-5e56078b6783 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:38.537941 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-db04f81c-8d3f-43d9-835a-5e56078b6783 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:38.538900 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-db04f81c-8d3f-43d9-835a-5e56078b6783 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:38.539667 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-db04f81c-8d3f-43d9-835a-5e56078b6783 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:38.540555 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-db04f81c-8d3f-43d9-835a-5e56078b6783 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:38.596209 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-06b1c3db-0fa4-4dee-826e-f258b10172a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:38.596879 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-06b1c3db-0fa4-4dee-826e-f258b10172a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:38.605956 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06b1c3db-0fa4-4dee-826e-f258b10172a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:38.605956 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06b1c3db-0fa4-4dee-826e-f258b10172a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:38.605956 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06b1c3db-0fa4-4dee-826e-f258b10172a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:38.605956 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06b1c3db-0fa4-4dee-826e-f258b10172a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:38.606508 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06b1c3db-0fa4-4dee-826e-f258b10172a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:38.606696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06b1c3db-0fa4-4dee-826e-f258b10172a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:38.633433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-db04f81c-8d3f-43d9-835a-5e56078b6783 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:38.635983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-db04f81c-8d3f-43d9-835a-5e56078b6783 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1223 microversion: 2.1 time: 0.116576 Aug 21 01:28:38.636629 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 222/439] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:38 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1223 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:38.755719 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-06b1c3db-0fa4-4dee-826e-f258b10172a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:38.766257 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-06b1c3db-0fa4-4dee-826e-f258b10172a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:38.767770 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-06b1c3db-0fa4-4dee-826e-f258b10172a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:38.769010 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-06b1c3db-0fa4-4dee-826e-f258b10172a4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.176866 Aug 21 01:28:38.769586 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 218/440] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:38 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:28:39.657319 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2654f22f-ef73-4204-8bbd-bad6d939dbe3 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:39.658041 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-2654f22f-ef73-4204-8bbd-bad6d939dbe3 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:39.664538 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2654f22f-ef73-4204-8bbd-bad6d939dbe3 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:39.665029 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2654f22f-ef73-4204-8bbd-bad6d939dbe3 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:39.665474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2654f22f-ef73-4204-8bbd-bad6d939dbe3 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:39.666204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2654f22f-ef73-4204-8bbd-bad6d939dbe3 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:39.666663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2654f22f-ef73-4204-8bbd-bad6d939dbe3 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:39.667093 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2654f22f-ef73-4204-8bbd-bad6d939dbe3 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:39.743465 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2654f22f-ef73-4204-8bbd-bad6d939dbe3 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:39.752073 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2654f22f-ef73-4204-8bbd-bad6d939dbe3 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1271 microversion: 2.1 time: 0.099540 Aug 21 01:28:39.752669 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 223/441] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:39 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1271 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:39.804444 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f5d2c331-0c18-4ed2-a194-48541f1aed9a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:39.805118 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f5d2c331-0c18-4ed2-a194-48541f1aed9a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:39.812389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f5d2c331-0c18-4ed2-a194-48541f1aed9a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:39.812877 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f5d2c331-0c18-4ed2-a194-48541f1aed9a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:39.813345 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f5d2c331-0c18-4ed2-a194-48541f1aed9a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:39.814044 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f5d2c331-0c18-4ed2-a194-48541f1aed9a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:39.814657 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f5d2c331-0c18-4ed2-a194-48541f1aed9a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:39.815173 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f5d2c331-0c18-4ed2-a194-48541f1aed9a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:40.038629 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f5d2c331-0c18-4ed2-a194-48541f1aed9a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:40.050476 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f5d2c331-0c18-4ed2-a194-48541f1aed9a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:40.052222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f5d2c331-0c18-4ed2-a194-48541f1aed9a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:40.053549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f5d2c331-0c18-4ed2-a194-48541f1aed9a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.253748 Aug 21 01:28:40.054216 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 219/442] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:39 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:28:40.774275 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3e446152-ad14-4b2d-9c6b-5416fa787161 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:40.775035 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3e446152-ad14-4b2d-9c6b-5416fa787161 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:40.782471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3e446152-ad14-4b2d-9c6b-5416fa787161 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:40.782966 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3e446152-ad14-4b2d-9c6b-5416fa787161 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:40.783402 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3e446152-ad14-4b2d-9c6b-5416fa787161 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:40.784132 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3e446152-ad14-4b2d-9c6b-5416fa787161 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:40.784602 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3e446152-ad14-4b2d-9c6b-5416fa787161 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:40.785005 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3e446152-ad14-4b2d-9c6b-5416fa787161 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:40.985985 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3e446152-ad14-4b2d-9c6b-5416fa787161 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:40.994685 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3e446152-ad14-4b2d-9c6b-5416fa787161 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1319 microversion: 2.1 time: 0.224160 Aug 21 01:28:40.995680 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 224/443] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:40 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1319 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:41.079637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-182102fa-ebfd-49be-86ed-e8d3c7e9933f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:41.080476 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-182102fa-ebfd-49be-86ed-e8d3c7e9933f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:41.087014 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-182102fa-ebfd-49be-86ed-e8d3c7e9933f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:41.087501 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-182102fa-ebfd-49be-86ed-e8d3c7e9933f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:41.087942 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-182102fa-ebfd-49be-86ed-e8d3c7e9933f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:41.088716 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-182102fa-ebfd-49be-86ed-e8d3c7e9933f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:41.089133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-182102fa-ebfd-49be-86ed-e8d3c7e9933f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:41.089560 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-182102fa-ebfd-49be-86ed-e8d3c7e9933f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:41.354144 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-182102fa-ebfd-49be-86ed-e8d3c7e9933f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:41.371210 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-182102fa-ebfd-49be-86ed-e8d3c7e9933f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:41.373862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-182102fa-ebfd-49be-86ed-e8d3c7e9933f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:41.375479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-182102fa-ebfd-49be-86ed-e8d3c7e9933f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.299813 Aug 21 01:28:41.376305 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 220/444] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:41 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:28:41.840841 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-43a1e938-5500-4d37-be4f-832d6bd7739b req-eed642a3-c6ea-41f9-9b38-4993e401d831 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "37524b56-3cc1-4276-802b-1419dee34d23", "tag": "23ea1b5c-1202-43ad-af9f-6cc34179339a"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:41.850780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-43a1e938-5500-4d37-be4f-832d6bd7739b req-eed642a3-c6ea-41f9-9b38-4993e401d831 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:41.851840 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-43a1e938-5500-4d37-be4f-832d6bd7739b req-eed642a3-c6ea-41f9-9b38-4993e401d831 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:41.851840 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-43a1e938-5500-4d37-be4f-832d6bd7739b req-eed642a3-c6ea-41f9-9b38-4993e401d831 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:41.863130 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-43a1e938-5500-4d37-be4f-832d6bd7739b req-eed642a3-c6ea-41f9-9b38-4993e401d831 service nova] Creating event network-changed:23ea1b5c-1202-43ad-af9f-6cc34179339a for instance 37524b56-3cc1-4276-802b-1419dee34d23 on hv-cinder-maste Aug 21 01:28:41.868343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-43a1e938-5500-4d37-be4f-832d6bd7739b req-eed642a3-c6ea-41f9-9b38-4993e401d831 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032746 Aug 21 01:28:41.868943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 225/445] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:28:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:28:42.023830 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d45f1253-0b2b-4d43-9a75-540403d79339 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:42.024481 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d45f1253-0b2b-4d43-9a75-540403d79339 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:42.032072 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d45f1253-0b2b-4d43-9a75-540403d79339 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:42.032834 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d45f1253-0b2b-4d43-9a75-540403d79339 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:42.033342 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d45f1253-0b2b-4d43-9a75-540403d79339 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:42.034202 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d45f1253-0b2b-4d43-9a75-540403d79339 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:42.034753 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d45f1253-0b2b-4d43-9a75-540403d79339 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:42.035555 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d45f1253-0b2b-4d43-9a75-540403d79339 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:42.189621 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d45f1253-0b2b-4d43-9a75-540403d79339 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:42.197442 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d45f1253-0b2b-4d43-9a75-540403d79339 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1319 microversion: 2.1 time: 0.177394 Aug 21 01:28:42.198004 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 221/446] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:42 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1319 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:42.404071 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-bcb99d22-fcb3-43a8-b9d0-5aeccc2688c2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:42.405095 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-bcb99d22-fcb3-43a8-b9d0-5aeccc2688c2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:42.413670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bcb99d22-fcb3-43a8-b9d0-5aeccc2688c2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:42.414301 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bcb99d22-fcb3-43a8-b9d0-5aeccc2688c2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:42.414835 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bcb99d22-fcb3-43a8-b9d0-5aeccc2688c2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:42.415643 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bcb99d22-fcb3-43a8-b9d0-5aeccc2688c2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:42.416260 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bcb99d22-fcb3-43a8-b9d0-5aeccc2688c2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:42.416858 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bcb99d22-fcb3-43a8-b9d0-5aeccc2688c2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:42.594684 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-bcb99d22-fcb3-43a8-b9d0-5aeccc2688c2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:42.610285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-bcb99d22-fcb3-43a8-b9d0-5aeccc2688c2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:42.612316 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-bcb99d22-fcb3-43a8-b9d0-5aeccc2688c2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:42.613706 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-bcb99d22-fcb3-43a8-b9d0-5aeccc2688c2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.215465 Aug 21 01:28:42.614320 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 226/447] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:42 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:43.218223 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7529b031-3ff6-4020-a15d-9492b3cb7a3a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:43.218834 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7529b031-3ff6-4020-a15d-9492b3cb7a3a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:43.225586 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7529b031-3ff6-4020-a15d-9492b3cb7a3a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:43.226075 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7529b031-3ff6-4020-a15d-9492b3cb7a3a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:43.226510 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7529b031-3ff6-4020-a15d-9492b3cb7a3a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:43.227210 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7529b031-3ff6-4020-a15d-9492b3cb7a3a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:43.227666 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7529b031-3ff6-4020-a15d-9492b3cb7a3a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:43.228092 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7529b031-3ff6-4020-a15d-9492b3cb7a3a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:43.256840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=100 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:43.461592 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7529b031-3ff6-4020-a15d-9492b3cb7a3a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:43.473293 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7529b031-3ff6-4020-a15d-9492b3cb7a3a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1319 microversion: 2.1 time: 0.258932 Aug 21 01:28:43.473874 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 222/448] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:43 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1319 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:43.657023 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-286bba4d-26b3-4b55-8c6e-8a4b27b3464b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:43.657595 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-286bba4d-26b3-4b55-8c6e-8a4b27b3464b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:43.660187 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:43.666650 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-286bba4d-26b3-4b55-8c6e-8a4b27b3464b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:43.666650 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-286bba4d-26b3-4b55-8c6e-8a4b27b3464b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:43.667229 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-286bba4d-26b3-4b55-8c6e-8a4b27b3464b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:43.668042 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-286bba4d-26b3-4b55-8c6e-8a4b27b3464b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:43.669380 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-286bba4d-26b3-4b55-8c6e-8a4b27b3464b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:43.669883 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-286bba4d-26b3-4b55-8c6e-8a4b27b3464b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:43.829612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-286bba4d-26b3-4b55-8c6e-8a4b27b3464b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:43.851011 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-286bba4d-26b3-4b55-8c6e-8a4b27b3464b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:43.853867 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-286bba4d-26b3-4b55-8c6e-8a4b27b3464b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:43.855883 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-286bba4d-26b3-4b55-8c6e-8a4b27b3464b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.203239 Aug 21 01:28:43.858370 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 227/449] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:43 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:28:44.376061 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a45e97d0-815b-453c-9c7a-d5cdbc9bd1f9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:44.376970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a45e97d0-815b-453c-9c7a-d5cdbc9bd1f9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:44.383511 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a45e97d0-815b-453c-9c7a-d5cdbc9bd1f9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:44.384143 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a45e97d0-815b-453c-9c7a-d5cdbc9bd1f9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:44.384870 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a45e97d0-815b-453c-9c7a-d5cdbc9bd1f9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:44.385355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a45e97d0-815b-453c-9c7a-d5cdbc9bd1f9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:44.385891 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a45e97d0-815b-453c-9c7a-d5cdbc9bd1f9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:44.386247 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a45e97d0-815b-453c-9c7a-d5cdbc9bd1f9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:44.494918 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-df2c0ec7-d75d-4de2-830c-912896743db7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:44.495670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-df2c0ec7-d75d-4de2-830c-912896743db7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:44.512029 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-df2c0ec7-d75d-4de2-830c-912896743db7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:44.513125 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-df2c0ec7-d75d-4de2-830c-912896743db7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:44.513570 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-df2c0ec7-d75d-4de2-830c-912896743db7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:44.514318 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-df2c0ec7-d75d-4de2-830c-912896743db7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:44.514724 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-df2c0ec7-d75d-4de2-830c-912896743db7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:44.515243 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-df2c0ec7-d75d-4de2-830c-912896743db7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:44.574979 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-a45e97d0-815b-453c-9c7a-d5cdbc9bd1f9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lazy-loading 'services' on Instance uuid c266a09a-3f81-4111-a987-b1f4be65cdc4 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:28:44.711330 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-df2c0ec7-d75d-4de2-830c-912896743db7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:44.725485 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-df2c0ec7-d75d-4de2-830c-912896743db7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1319 microversion: 2.1 time: 0.234707 Aug 21 01:28:44.726423 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 228/450] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:44 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1319 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:44.886552 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-79fe9ed4-1ebf-40e3-97a2-b97c933e3bda tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:44.886831 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-79fe9ed4-1ebf-40e3-97a2-b97c933e3bda tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:44.893901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79fe9ed4-1ebf-40e3-97a2-b97c933e3bda tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:44.895125 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79fe9ed4-1ebf-40e3-97a2-b97c933e3bda tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:44.895125 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79fe9ed4-1ebf-40e3-97a2-b97c933e3bda tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:44.895670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79fe9ed4-1ebf-40e3-97a2-b97c933e3bda tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:44.896318 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79fe9ed4-1ebf-40e3-97a2-b97c933e3bda tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:44.896833 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79fe9ed4-1ebf-40e3-97a2-b97c933e3bda tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:44.933847 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a45e97d0-815b-453c-9c7a-d5cdbc9bd1f9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "DELETE /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/os-volume_attachments/bb48bf98-8498-4b4f-bd2a-cbfdc1713fed" status: 202 len: 0 microversion: 2.1 time: 0.561879 Aug 21 01:28:44.934909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 223/451] 10.222.0.47 () {58 vars in 1416 bytes} [Sun Aug 21 01:28:44 2022] DELETE /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/os-volume_attachments/bb48bf98-8498-4b4f-bd2a-cbfdc1713fed => generated 0 bytes in 565 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:28:45.112332 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-79fe9ed4-1ebf-40e3-97a2-b97c933e3bda tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:45.122011 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-79fe9ed4-1ebf-40e3-97a2-b97c933e3bda tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:45.123663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-79fe9ed4-1ebf-40e3-97a2-b97c933e3bda tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:45.125377 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-79fe9ed4-1ebf-40e3-97a2-b97c933e3bda tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.247727 Aug 21 01:28:45.126151 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 229/452] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:44 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:45.749254 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6641bae5-0e4d-4018-b9eb-42dabeda5d40 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:45.749949 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6641bae5-0e4d-4018-b9eb-42dabeda5d40 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:45.757180 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6641bae5-0e4d-4018-b9eb-42dabeda5d40 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:45.757802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6641bae5-0e4d-4018-b9eb-42dabeda5d40 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:45.758252 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6641bae5-0e4d-4018-b9eb-42dabeda5d40 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:45.759134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6641bae5-0e4d-4018-b9eb-42dabeda5d40 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:45.759796 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6641bae5-0e4d-4018-b9eb-42dabeda5d40 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:45.760367 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6641bae5-0e4d-4018-b9eb-42dabeda5d40 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:45.923696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6641bae5-0e4d-4018-b9eb-42dabeda5d40 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:45.941898 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6641bae5-0e4d-4018-b9eb-42dabeda5d40 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1319 microversion: 2.1 time: 0.196069 Aug 21 01:28:45.941898 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 224/453] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:45 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1319 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:46.149000 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d49e00e9-2ba1-4fb5-a21e-306733a63e37 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:46.149701 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d49e00e9-2ba1-4fb5-a21e-306733a63e37 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:46.160407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d49e00e9-2ba1-4fb5-a21e-306733a63e37 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:46.161748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d49e00e9-2ba1-4fb5-a21e-306733a63e37 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:46.161948 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d49e00e9-2ba1-4fb5-a21e-306733a63e37 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:46.162753 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d49e00e9-2ba1-4fb5-a21e-306733a63e37 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:46.163514 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d49e00e9-2ba1-4fb5-a21e-306733a63e37 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:46.164119 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d49e00e9-2ba1-4fb5-a21e-306733a63e37 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:46.357446 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-98e1d5fd-39c8-4bd5-b814-b6f6a8c8eeb8 req-63694a92-0a02-4aed-b52c-e6d3f9df0276 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1", "name": "network-vif-plugged", "status": "completed", "tag": "9a09b4f3-32e5-4362-82ef-ab73b634b1bf"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:46.368203 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-98e1d5fd-39c8-4bd5-b814-b6f6a8c8eeb8 req-63694a92-0a02-4aed-b52c-e6d3f9df0276 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:46.368761 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-98e1d5fd-39c8-4bd5-b814-b6f6a8c8eeb8 req-63694a92-0a02-4aed-b52c-e6d3f9df0276 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:46.369260 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-98e1d5fd-39c8-4bd5-b814-b6f6a8c8eeb8 req-63694a92-0a02-4aed-b52c-e6d3f9df0276 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:46.384207 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d49e00e9-2ba1-4fb5-a21e-306733a63e37 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:46.389624 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-98e1d5fd-39c8-4bd5-b814-b6f6a8c8eeb8 req-63694a92-0a02-4aed-b52c-e6d3f9df0276 service nova] Creating event network-vif-plugged:9a09b4f3-32e5-4362-82ef-ab73b634b1bf for instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 on hv-cinder-maste Aug 21 01:28:46.396955 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-98e1d5fd-39c8-4bd5-b814-b6f6a8c8eeb8 req-63694a92-0a02-4aed-b52c-e6d3f9df0276 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044202 Aug 21 01:28:46.398809 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 225/454] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:28:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:28:46.404204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d49e00e9-2ba1-4fb5-a21e-306733a63e37 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:46.404204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d49e00e9-2ba1-4fb5-a21e-306733a63e37 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:46.405305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d49e00e9-2ba1-4fb5-a21e-306733a63e37 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.260361 Aug 21 01:28:46.405869 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 230/455] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:46 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:46.964860 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f99aa0d0-503a-4e03-a2b1-57c692495c9d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:46.966184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f99aa0d0-503a-4e03-a2b1-57c692495c9d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:46.976960 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f99aa0d0-503a-4e03-a2b1-57c692495c9d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:46.977452 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f99aa0d0-503a-4e03-a2b1-57c692495c9d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:46.977893 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f99aa0d0-503a-4e03-a2b1-57c692495c9d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:46.978607 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f99aa0d0-503a-4e03-a2b1-57c692495c9d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:46.979067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f99aa0d0-503a-4e03-a2b1-57c692495c9d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:46.989001 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f99aa0d0-503a-4e03-a2b1-57c692495c9d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:47.111125 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:47.140225 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=10 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:47.158591 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f99aa0d0-503a-4e03-a2b1-57c692495c9d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:47.169897 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f99aa0d0-503a-4e03-a2b1-57c692495c9d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1319 microversion: 2.1 time: 0.211068 Aug 21 01:28:47.180493 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 226/456] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:46 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1319 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:47.265983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7511733c-cb7b-426a-aa3e-fba41256ee02 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:47.266731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7511733c-cb7b-426a-aa3e-fba41256ee02 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:47.273962 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7511733c-cb7b-426a-aa3e-fba41256ee02 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:47.274478 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7511733c-cb7b-426a-aa3e-fba41256ee02 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:47.274972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7511733c-cb7b-426a-aa3e-fba41256ee02 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:47.275667 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7511733c-cb7b-426a-aa3e-fba41256ee02 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:47.276147 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7511733c-cb7b-426a-aa3e-fba41256ee02 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:47.276573 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7511733c-cb7b-426a-aa3e-fba41256ee02 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:47.443192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4f6cc827-b6d9-456e-b0f1-1c686384dd3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:47.444488 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4f6cc827-b6d9-456e-b0f1-1c686384dd3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:47.455115 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4f6cc827-b6d9-456e-b0f1-1c686384dd3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:47.455619 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4f6cc827-b6d9-456e-b0f1-1c686384dd3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:47.456069 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4f6cc827-b6d9-456e-b0f1-1c686384dd3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:47.456512 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7511733c-cb7b-426a-aa3e-fba41256ee02 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:47.456816 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4f6cc827-b6d9-456e-b0f1-1c686384dd3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:47.457305 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4f6cc827-b6d9-456e-b0f1-1c686384dd3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:47.457739 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4f6cc827-b6d9-456e-b0f1-1c686384dd3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:47.470276 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7511733c-cb7b-426a-aa3e-fba41256ee02 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1594 microversion: 2.1 time: 0.211387 Aug 21 01:28:47.471721 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 231/457] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:47 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1594 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:47.632514 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4f6cc827-b6d9-456e-b0f1-1c686384dd3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:47.642201 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4f6cc827-b6d9-456e-b0f1-1c686384dd3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:47.643663 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4f6cc827-b6d9-456e-b0f1-1c686384dd3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:47.644938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4f6cc827-b6d9-456e-b0f1-1c686384dd3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.219402 Aug 21 01:28:47.645498 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 227/458] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:47 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:48.198653 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e63d8d86-3b93-4236-9277-3892a223a712 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:48.201040 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e63d8d86-3b93-4236-9277-3892a223a712 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:48.224286 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:48.225685 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=54 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:48.239538 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e63d8d86-3b93-4236-9277-3892a223a712 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:48.243733 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e63d8d86-3b93-4236-9277-3892a223a712 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:48.244883 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e63d8d86-3b93-4236-9277-3892a223a712 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:48.248645 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e63d8d86-3b93-4236-9277-3892a223a712 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:48.248645 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e63d8d86-3b93-4236-9277-3892a223a712 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:48.248645 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e63d8d86-3b93-4236-9277-3892a223a712 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:48.353024 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-dbd0bd3d-3001-475f-acea-13ba6b15a0ef tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:48.362484 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbd0bd3d-3001-475f-acea-13ba6b15a0ef tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:48.363330 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbd0bd3d-3001-475f-acea-13ba6b15a0ef tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:48.364062 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbd0bd3d-3001-475f-acea-13ba6b15a0ef tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:48.399813 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-dbd0bd3d-3001-475f-acea-13ba6b15a0ef tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Going to try to stop instance {{(pid=121276) force_stop /opt/stack/nova/nova/compute/api.py:2749}} Aug 21 01:28:48.489072 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e63d8d86-3b93-4236-9277-3892a223a712 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:48.504212 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e63d8d86-3b93-4236-9277-3892a223a712 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1319 microversion: 2.1 time: 0.311684 Aug 21 01:28:48.508201 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 232/459] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:48 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1319 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:48.616075 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-dbd0bd3d-3001-475f-acea-13ba6b15a0ef tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "POST /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/action" status: 202 len: 0 microversion: 2.1 time: 0.266690 Aug 21 01:28:48.616743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 228/460] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:28:48 2022] POST /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/action => generated 0 bytes in 268 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:28:48.625705 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6115a1e4-f435-4933-a0bc-4d8cd53eef6e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:48.626340 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6115a1e4-f435-4933-a0bc-4d8cd53eef6e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:48.632429 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6115a1e4-f435-4933-a0bc-4d8cd53eef6e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:48.632954 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6115a1e4-f435-4933-a0bc-4d8cd53eef6e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:48.633389 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6115a1e4-f435-4933-a0bc-4d8cd53eef6e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:48.634070 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6115a1e4-f435-4933-a0bc-4d8cd53eef6e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:48.634522 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6115a1e4-f435-4933-a0bc-4d8cd53eef6e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:48.634945 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6115a1e4-f435-4933-a0bc-4d8cd53eef6e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:48.668404 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-61e25851-5ecc-4232-bac2-eed35b981b02 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:48.670234 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-61e25851-5ecc-4232-bac2-eed35b981b02 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:48.677076 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-61e25851-5ecc-4232-bac2-eed35b981b02 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:48.677560 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-61e25851-5ecc-4232-bac2-eed35b981b02 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:48.678562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-61e25851-5ecc-4232-bac2-eed35b981b02 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:48.680074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-61e25851-5ecc-4232-bac2-eed35b981b02 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:48.681043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-61e25851-5ecc-4232-bac2-eed35b981b02 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:48.681799 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-61e25851-5ecc-4232-bac2-eed35b981b02 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:48.875075 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6115a1e4-f435-4933-a0bc-4d8cd53eef6e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:48.889237 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6115a1e4-f435-4933-a0bc-4d8cd53eef6e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1725 microversion: 2.1 time: 0.267744 Aug 21 01:28:48.899671 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 233/461] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:48 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1725 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:48.913415 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-61e25851-5ecc-4232-bac2-eed35b981b02 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:48.937329 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-61e25851-5ecc-4232-bac2-eed35b981b02 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:48.941614 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-61e25851-5ecc-4232-bac2-eed35b981b02 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:48.944565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-61e25851-5ecc-4232-bac2-eed35b981b02 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.280480 Aug 21 01:28:48.945829 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 229/462] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:48 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:49.546587 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2bf6b6a0-9866-4faf-80ac-d03ab3735a5b tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:49.547390 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-2bf6b6a0-9866-4faf-80ac-d03ab3735a5b tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:49.554797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2bf6b6a0-9866-4faf-80ac-d03ab3735a5b tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:49.555307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2bf6b6a0-9866-4faf-80ac-d03ab3735a5b tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:49.555951 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2bf6b6a0-9866-4faf-80ac-d03ab3735a5b tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:49.556700 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2bf6b6a0-9866-4faf-80ac-d03ab3735a5b tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:49.557284 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2bf6b6a0-9866-4faf-80ac-d03ab3735a5b tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:49.557715 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2bf6b6a0-9866-4faf-80ac-d03ab3735a5b tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:49.755118 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2bf6b6a0-9866-4faf-80ac-d03ab3735a5b tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:49.765218 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2bf6b6a0-9866-4faf-80ac-d03ab3735a5b tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1319 microversion: 2.1 time: 0.223229 Aug 21 01:28:49.765801 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 234/463] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:49 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1319 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:49.913852 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3d452be2-01c9-412d-bb19-3e8cfefc44c6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:49.915492 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3d452be2-01c9-412d-bb19-3e8cfefc44c6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:49.924621 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3d452be2-01c9-412d-bb19-3e8cfefc44c6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:49.925355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3d452be2-01c9-412d-bb19-3e8cfefc44c6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:49.926093 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3d452be2-01c9-412d-bb19-3e8cfefc44c6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:49.927061 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3d452be2-01c9-412d-bb19-3e8cfefc44c6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:49.927731 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3d452be2-01c9-412d-bb19-3e8cfefc44c6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:49.928372 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3d452be2-01c9-412d-bb19-3e8cfefc44c6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:49.980780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f69d74a9-2cb7-40f9-bf93-b4f029a7b181 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:49.982163 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f69d74a9-2cb7-40f9-bf93-b4f029a7b181 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:49.994508 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f69d74a9-2cb7-40f9-bf93-b4f029a7b181 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:49.995374 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f69d74a9-2cb7-40f9-bf93-b4f029a7b181 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:49.996191 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f69d74a9-2cb7-40f9-bf93-b4f029a7b181 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:49.997191 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f69d74a9-2cb7-40f9-bf93-b4f029a7b181 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:49.998694 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f69d74a9-2cb7-40f9-bf93-b4f029a7b181 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:49.999638 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f69d74a9-2cb7-40f9-bf93-b4f029a7b181 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:50.176981 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3d452be2-01c9-412d-bb19-3e8cfefc44c6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:50.186802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3d452be2-01c9-412d-bb19-3e8cfefc44c6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.276747 Aug 21 01:28:50.188230 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 230/464] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:49 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:50.219831 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f69d74a9-2cb7-40f9-bf93-b4f029a7b181 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:50.223368 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:50.232231 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f69d74a9-2cb7-40f9-bf93-b4f029a7b181 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:50.232231 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f69d74a9-2cb7-40f9-bf93-b4f029a7b181 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:50.233416 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f69d74a9-2cb7-40f9-bf93-b4f029a7b181 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.258066 Aug 21 01:28:50.233974 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 235/465] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:49 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:28:50.333874 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e1bc7b6f-ce6e-4efc-9dcf-eaf2f51c0ba1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2fb97ee5-b984-45f9-8ae3-f2209e4b20e3"}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:50.338029 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e1bc7b6f-ce6e-4efc-9dcf-eaf2f51c0ba1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:50.340487 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-9a6f7a5f-095d-4c38-8e27-39714ceb4a45 req-57a54a5b-b432-4ffb-b91e-bb2e849ad204 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9bc908f3-d8e3-4432-8197-51096e59a470", "tag": "9b282ab5-9500-4319-bbaa-ac3078e3128d"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:50.350970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e1bc7b6f-ce6e-4efc-9dcf-eaf2f51c0ba1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:50.356219 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e1bc7b6f-ce6e-4efc-9dcf-eaf2f51c0ba1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:50.356219 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e1bc7b6f-ce6e-4efc-9dcf-eaf2f51c0ba1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:50.356219 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e1bc7b6f-ce6e-4efc-9dcf-eaf2f51c0ba1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:50.356219 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e1bc7b6f-ce6e-4efc-9dcf-eaf2f51c0ba1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:50.358259 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e1bc7b6f-ce6e-4efc-9dcf-eaf2f51c0ba1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:50.359899 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9a6f7a5f-095d-4c38-8e27-39714ceb4a45 req-57a54a5b-b432-4ffb-b91e-bb2e849ad204 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:50.360420 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9a6f7a5f-095d-4c38-8e27-39714ceb4a45 req-57a54a5b-b432-4ffb-b91e-bb2e849ad204 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:50.360911 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9a6f7a5f-095d-4c38-8e27-39714ceb4a45 req-57a54a5b-b432-4ffb-b91e-bb2e849ad204 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:50.375258 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-9a6f7a5f-095d-4c38-8e27-39714ceb4a45 req-57a54a5b-b432-4ffb-b91e-bb2e849ad204 service nova] Creating event network-changed:9b282ab5-9500-4319-bbaa-ac3078e3128d for instance 9bc908f3-d8e3-4432-8197-51096e59a470 on hv-cinder-maste Aug 21 01:28:50.381683 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-9a6f7a5f-095d-4c38-8e27-39714ceb4a45 req-57a54a5b-b432-4ffb-b91e-bb2e849ad204 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046614 Aug 21 01:28:50.382926 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 236/466] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:28:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:28:50.791906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7119898a-d79a-4872-bda9-85cc1300aba5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:50.793817 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7119898a-d79a-4872-bda9-85cc1300aba5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:50.823617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7119898a-d79a-4872-bda9-85cc1300aba5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:50.826940 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7119898a-d79a-4872-bda9-85cc1300aba5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:50.829084 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7119898a-d79a-4872-bda9-85cc1300aba5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:50.830155 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7119898a-d79a-4872-bda9-85cc1300aba5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:50.831654 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7119898a-d79a-4872-bda9-85cc1300aba5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:50.832277 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7119898a-d79a-4872-bda9-85cc1300aba5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:51.423966 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7119898a-d79a-4872-bda9-85cc1300aba5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:51.431854 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7119898a-d79a-4872-bda9-85cc1300aba5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1319 microversion: 2.1 time: 0.648272 Aug 21 01:28:51.432514 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 237/467] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:50 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1319 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:51.437325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e0323d38-f5d1-44d2-ad8b-f0c7c5c0ba8e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:51.437943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e0323d38-f5d1-44d2-ad8b-f0c7c5c0ba8e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:51.447051 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e1bc7b6f-ce6e-4efc-9dcf-eaf2f51c0ba1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "POST /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.120273 Aug 21 01:28:51.449218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 231/468] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:28:50 2022] POST /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments => generated 194 bytes in 1123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:28:51.459532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6ea3c687-1dee-4852-92af-44c90e1d13c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:51.461368 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6ea3c687-1dee-4852-92af-44c90e1d13c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:51.467998 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e0323d38-f5d1-44d2-ad8b-f0c7c5c0ba8e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:51.471957 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e0323d38-f5d1-44d2-ad8b-f0c7c5c0ba8e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:51.472757 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e0323d38-f5d1-44d2-ad8b-f0c7c5c0ba8e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:51.474637 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e0323d38-f5d1-44d2-ad8b-f0c7c5c0ba8e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:51.475270 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e0323d38-f5d1-44d2-ad8b-f0c7c5c0ba8e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:51.476454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e0323d38-f5d1-44d2-ad8b-f0c7c5c0ba8e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:51.483600 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6ea3c687-1dee-4852-92af-44c90e1d13c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:51.484327 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6ea3c687-1dee-4852-92af-44c90e1d13c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:51.484783 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6ea3c687-1dee-4852-92af-44c90e1d13c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:51.485522 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6ea3c687-1dee-4852-92af-44c90e1d13c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:51.485970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6ea3c687-1dee-4852-92af-44c90e1d13c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:51.486921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6ea3c687-1dee-4852-92af-44c90e1d13c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:51.840795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e0323d38-f5d1-44d2-ad8b-f0c7c5c0ba8e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:51.846141 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6ea3c687-1dee-4852-92af-44c90e1d13c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:51.858626 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e0323d38-f5d1-44d2-ad8b-f0c7c5c0ba8e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.424927 Aug 21 01:28:51.859112 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 238/469] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:51 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:51.863632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6ea3c687-1dee-4852-92af-44c90e1d13c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:51.867649 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6ea3c687-1dee-4852-92af-44c90e1d13c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:51.870166 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-50f2fff1-3d7f-47eb-acd2-88f92a406a8f req-5c9fd580-abdf-429f-8ad5-200885f97f7f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "37524b56-3cc1-4276-802b-1419dee34d23", "name": "network-vif-plugged", "status": "completed", "tag": "23ea1b5c-1202-43ad-af9f-6cc34179339a"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:51.871866 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6ea3c687-1dee-4852-92af-44c90e1d13c8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1807 microversion: 2.20 time: 0.420893 Aug 21 01:28:51.874070 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 232/470] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:51 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1807 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:51.885516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-50f2fff1-3d7f-47eb-acd2-88f92a406a8f req-5c9fd580-abdf-429f-8ad5-200885f97f7f service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:51.885993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-50f2fff1-3d7f-47eb-acd2-88f92a406a8f req-5c9fd580-abdf-429f-8ad5-200885f97f7f service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:51.886427 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-50f2fff1-3d7f-47eb-acd2-88f92a406a8f req-5c9fd580-abdf-429f-8ad5-200885f97f7f service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:51.899813 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-50f2fff1-3d7f-47eb-acd2-88f92a406a8f req-5c9fd580-abdf-429f-8ad5-200885f97f7f service nova] Creating event network-vif-plugged:23ea1b5c-1202-43ad-af9f-6cc34179339a for instance 37524b56-3cc1-4276-802b-1419dee34d23 on hv-cinder-maste Aug 21 01:28:51.905061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-50f2fff1-3d7f-47eb-acd2-88f92a406a8f req-5c9fd580-abdf-429f-8ad5-200885f97f7f service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044646 Aug 21 01:28:51.905524 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 239/471] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:28:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:28:52.464815 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-908e5ebc-42d7-4e6a-b2ed-1f9b36fa2687 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:52.465466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-908e5ebc-42d7-4e6a-b2ed-1f9b36fa2687 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:52.472430 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-908e5ebc-42d7-4e6a-b2ed-1f9b36fa2687 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:52.472926 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-908e5ebc-42d7-4e6a-b2ed-1f9b36fa2687 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:52.473371 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-908e5ebc-42d7-4e6a-b2ed-1f9b36fa2687 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:52.474102 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-908e5ebc-42d7-4e6a-b2ed-1f9b36fa2687 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:52.474549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-908e5ebc-42d7-4e6a-b2ed-1f9b36fa2687 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:52.475088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-908e5ebc-42d7-4e6a-b2ed-1f9b36fa2687 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:52.647881 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-908e5ebc-42d7-4e6a-b2ed-1f9b36fa2687 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:52.658010 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-908e5ebc-42d7-4e6a-b2ed-1f9b36fa2687 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1319 microversion: 2.1 time: 0.197060 Aug 21 01:28:52.658709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 233/472] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:52 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1319 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:52.898803 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7b12c262-9294-48f8-84cc-a0437a9da3e0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:52.899547 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7b12c262-9294-48f8-84cc-a0437a9da3e0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:52.901038 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-cb322cea-6d30-42b0-8b1e-25b0df93e58c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:52.902357 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-cb322cea-6d30-42b0-8b1e-25b0df93e58c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:52.907808 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7b12c262-9294-48f8-84cc-a0437a9da3e0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:52.908670 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7b12c262-9294-48f8-84cc-a0437a9da3e0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:52.909192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7b12c262-9294-48f8-84cc-a0437a9da3e0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:52.909975 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7b12c262-9294-48f8-84cc-a0437a9da3e0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:52.910444 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7b12c262-9294-48f8-84cc-a0437a9da3e0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:52.910878 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7b12c262-9294-48f8-84cc-a0437a9da3e0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:52.913677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb322cea-6d30-42b0-8b1e-25b0df93e58c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:52.914388 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb322cea-6d30-42b0-8b1e-25b0df93e58c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:52.915178 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb322cea-6d30-42b0-8b1e-25b0df93e58c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:52.916212 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb322cea-6d30-42b0-8b1e-25b0df93e58c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:52.916810 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb322cea-6d30-42b0-8b1e-25b0df93e58c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:52.917260 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb322cea-6d30-42b0-8b1e-25b0df93e58c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:53.111068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7b12c262-9294-48f8-84cc-a0437a9da3e0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:53.121473 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-cb322cea-6d30-42b0-8b1e-25b0df93e58c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:53.128420 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7b12c262-9294-48f8-84cc-a0437a9da3e0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:53.129781 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-cb322cea-6d30-42b0-8b1e-25b0df93e58c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.237501 Aug 21 01:28:53.130549 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 240/473] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:52 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:53.133316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7b12c262-9294-48f8-84cc-a0437a9da3e0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:53.134538 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7b12c262-9294-48f8-84cc-a0437a9da3e0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1794 microversion: 2.20 time: 0.239851 Aug 21 01:28:53.135109 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 234/474] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:28:52 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1794 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:28:53.453185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-24d9f76e-6ba0-496e-93f4-33341e6524ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1575003272"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:53.453729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-24d9f76e-6ba0-496e-93f4-33341e6524ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:53.459821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-24d9f76e-6ba0-496e-93f4-33341e6524ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:53.460347 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-24d9f76e-6ba0-496e-93f4-33341e6524ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:53.460787 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-24d9f76e-6ba0-496e-93f4-33341e6524ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:53.461488 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-24d9f76e-6ba0-496e-93f4-33341e6524ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:53.461943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-24d9f76e-6ba0-496e-93f4-33341e6524ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:53.462368 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-24d9f76e-6ba0-496e-93f4-33341e6524ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:53.608190 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.network.security_group_api [None req-24d9f76e-6ba0-496e-93f4-33341e6524ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Adding security group 81fe8dd3-7ec5-4c04-b2b5-ba8178555d1c to port 9b282ab5-9500-4319-bbaa-ac3078e3128d Aug 21 01:28:53.679734 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f8f3b84f-53ce-40b0-908d-24f548ed76cf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:53.680510 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f8f3b84f-53ce-40b0-908d-24f548ed76cf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:53.691108 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f8f3b84f-53ce-40b0-908d-24f548ed76cf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:53.691656 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f8f3b84f-53ce-40b0-908d-24f548ed76cf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:53.692241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f8f3b84f-53ce-40b0-908d-24f548ed76cf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:53.693312 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f8f3b84f-53ce-40b0-908d-24f548ed76cf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:53.694090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f8f3b84f-53ce-40b0-908d-24f548ed76cf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:53.695067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f8f3b84f-53ce-40b0-908d-24f548ed76cf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:53.915529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f8f3b84f-53ce-40b0-908d-24f548ed76cf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:53.924724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f8f3b84f-53ce-40b0-908d-24f548ed76cf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1319 microversion: 2.1 time: 0.248780 Aug 21 01:28:53.925315 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 235/475] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:53 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1319 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:54.148363 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-24d9f76e-6ba0-496e-93f4-33341e6524ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "POST /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/action" status: 202 len: 0 microversion: 2.1 time: 0.699980 Aug 21 01:28:54.149080 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 241/476] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:28:53 2022] POST /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/action => generated 0 bytes in 702 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:28:54.159256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d1ad480b-1b63-41b6-8812-db9bce09f0ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:54.159981 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d1ad480b-1b63-41b6-8812-db9bce09f0ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:54.162341 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-2fd7f29e-341c-45e6-ae68-419188f7f98f req-ca068c3b-fd58-444b-9747-0f3b4ccaa142 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9bc908f3-d8e3-4432-8197-51096e59a470", "tag": "9b282ab5-9500-4319-bbaa-ac3078e3128d"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:54.167088 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1ad480b-1b63-41b6-8812-db9bce09f0ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:54.167585 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1ad480b-1b63-41b6-8812-db9bce09f0ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:54.170630 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1ad480b-1b63-41b6-8812-db9bce09f0ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:54.170630 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1ad480b-1b63-41b6-8812-db9bce09f0ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:54.170630 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1ad480b-1b63-41b6-8812-db9bce09f0ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:54.170630 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1ad480b-1b63-41b6-8812-db9bce09f0ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:54.177086 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2fd7f29e-341c-45e6-ae68-419188f7f98f req-ca068c3b-fd58-444b-9747-0f3b4ccaa142 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:54.177306 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2fd7f29e-341c-45e6-ae68-419188f7f98f req-ca068c3b-fd58-444b-9747-0f3b4ccaa142 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:54.177577 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2fd7f29e-341c-45e6-ae68-419188f7f98f req-ca068c3b-fd58-444b-9747-0f3b4ccaa142 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:54.192181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-2fd7f29e-341c-45e6-ae68-419188f7f98f req-ca068c3b-fd58-444b-9747-0f3b4ccaa142 service nova] Creating event network-changed:9b282ab5-9500-4319-bbaa-ac3078e3128d for instance 9bc908f3-d8e3-4432-8197-51096e59a470 on hv-cinder-maste Aug 21 01:28:54.197808 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-2fd7f29e-341c-45e6-ae68-419188f7f98f req-ca068c3b-fd58-444b-9747-0f3b4ccaa142 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044183 Aug 21 01:28:54.198744 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 236/477] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:28:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:28:54.204401 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1eca56d6-ceb6-474e-9ac2-3be6add8df55 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:54.205109 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1eca56d6-ceb6-474e-9ac2-3be6add8df55 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:54.212297 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eca56d6-ceb6-474e-9ac2-3be6add8df55 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:54.213022 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eca56d6-ceb6-474e-9ac2-3be6add8df55 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:54.213468 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eca56d6-ceb6-474e-9ac2-3be6add8df55 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:54.214182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eca56d6-ceb6-474e-9ac2-3be6add8df55 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:54.214646 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eca56d6-ceb6-474e-9ac2-3be6add8df55 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:54.215076 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eca56d6-ceb6-474e-9ac2-3be6add8df55 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:54.243171 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=3,nova_cell1:UPDATE=3 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:28:54.442373 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d1ad480b-1b63-41b6-8812-db9bce09f0ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:54.449801 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1eca56d6-ceb6-474e-9ac2-3be6add8df55 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:54.452059 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d1ad480b-1b63-41b6-8812-db9bce09f0ba tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1760 microversion: 2.1 time: 0.296482 Aug 21 01:28:54.457401 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 242/478] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:54 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1760 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:54.459951 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1eca56d6-ceb6-474e-9ac2-3be6add8df55 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.260012 Aug 21 01:28:54.460711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 237/479] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:54 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:54.948735 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-08ac0902-2d15-4919-8d34-dd4e4a61d904 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:54.949824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-08ac0902-2d15-4919-8d34-dd4e4a61d904 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:54.956810 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-08ac0902-2d15-4919-8d34-dd4e4a61d904 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:54.957306 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-08ac0902-2d15-4919-8d34-dd4e4a61d904 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:54.957743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-08ac0902-2d15-4919-8d34-dd4e4a61d904 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:54.958449 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-08ac0902-2d15-4919-8d34-dd4e4a61d904 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:54.959173 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-08ac0902-2d15-4919-8d34-dd4e4a61d904 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:54.959672 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-08ac0902-2d15-4919-8d34-dd4e4a61d904 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:55.191086 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-08ac0902-2d15-4919-8d34-dd4e4a61d904 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:55.199235 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-08ac0902-2d15-4919-8d34-dd4e4a61d904 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1509 microversion: 2.1 time: 0.255785 Aug 21 01:28:55.200222 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 243/480] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:54 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1509 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:55.481202 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-81257bb8-ab68-43cc-863a-1dd1cb9e46f2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:55.481815 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-81257bb8-ab68-43cc-863a-1dd1cb9e46f2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:55.489667 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81257bb8-ab68-43cc-863a-1dd1cb9e46f2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:55.490227 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81257bb8-ab68-43cc-863a-1dd1cb9e46f2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:55.490667 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81257bb8-ab68-43cc-863a-1dd1cb9e46f2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:55.491635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81257bb8-ab68-43cc-863a-1dd1cb9e46f2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:55.492096 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81257bb8-ab68-43cc-863a-1dd1cb9e46f2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:55.492591 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81257bb8-ab68-43cc-863a-1dd1cb9e46f2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:55.664486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-81257bb8-ab68-43cc-863a-1dd1cb9e46f2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:55.672324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-81257bb8-ab68-43cc-863a-1dd1cb9e46f2 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.194920 Aug 21 01:28:55.672898 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 238/481] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:55 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:56.695332 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-097bdcaf-d672-48ff-9288-eb568ef89e85 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:56.695945 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-097bdcaf-d672-48ff-9288-eb568ef89e85 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:56.703632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-097bdcaf-d672-48ff-9288-eb568ef89e85 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:56.704153 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-097bdcaf-d672-48ff-9288-eb568ef89e85 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:56.704599 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-097bdcaf-d672-48ff-9288-eb568ef89e85 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:56.705293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-097bdcaf-d672-48ff-9288-eb568ef89e85 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:56.705746 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-097bdcaf-d672-48ff-9288-eb568ef89e85 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:56.706170 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-097bdcaf-d672-48ff-9288-eb568ef89e85 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:56.853720 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-097bdcaf-d672-48ff-9288-eb568ef89e85 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:56.862116 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-097bdcaf-d672-48ff-9288-eb568ef89e85 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.171067 Aug 21 01:28:56.863108 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 244/482] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:56 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:28:56.982700 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1601751578", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-281379194", "security_groups": [{"name": "tempest-secgroup-smoke-1289635027"}], "networks": [{"uuid": "e9956e3f-5992-4fc9-97b8-cb74fbffeffb"}], "block_device_mapping_v2": [{"uuid": "752bdc7d-8672-4d64-88ed-6192fc175341", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:28:57.124014 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] validate_networks() for [('e9956e3f-5992-4fc9-97b8-cb74fbffeffb', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:28:57.527033 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:28:57.527450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:28:57.527917 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:28:57.528377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:28:57.528786 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:28:57.538152 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:57.538655 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:57.539110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:57.539824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:57.540468 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:57.540861 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:57.566670 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Getting quotas for project ac06ecc6fc93477ba4909284525df514. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:28:57.570945 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Getting quotas for user 03d34d1e66404d259599806d040628ef and project ac06ecc6fc93477ba4909284525df514. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:28:57.633830 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:28:57.636462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='752bdc7d-8672-4d64-88ed-6192fc175341',volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:28:57.887704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9cbf1859-f1ce-471b-a780-47df7cb99666 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:57.889006 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9cbf1859-f1ce-471b-a780-47df7cb99666 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:57.898639 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9cbf1859-f1ce-471b-a780-47df7cb99666 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:57.899091 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9cbf1859-f1ce-471b-a780-47df7cb99666 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:57.899500 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9cbf1859-f1ce-471b-a780-47df7cb99666 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:57.900244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9cbf1859-f1ce-471b-a780-47df7cb99666 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:57.900702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9cbf1859-f1ce-471b-a780-47df7cb99666 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:57.901139 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9cbf1859-f1ce-471b-a780-47df7cb99666 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:58.045707 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9cbf1859-f1ce-471b-a780-47df7cb99666 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:58.053600 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9cbf1859-f1ce-471b-a780-47df7cb99666 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.170097 Aug 21 01:28:58.054290 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 245/483] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:57 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:58.146616 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8c049260-85cd-49bd-b32c-cbf23f0395ff tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.168146 Aug 21 01:28:58.151521 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 239/484] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:28:56 2022] POST /compute/v2.1/servers => generated 406 bytes in 1174 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:28:58.159884 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c25e1775-255a-42df-a507-cbf0b6748552 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:58.160533 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c25e1775-255a-42df-a507-cbf0b6748552 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:58.218396 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c25e1775-255a-42df-a507-cbf0b6748552 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:58.225626 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c25e1775-255a-42df-a507-cbf0b6748552 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1088 microversion: 2.1 time: 0.071196 Aug 21 01:28:58.228046 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 246/485] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:58 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1088 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:59.076864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7174a010-8c5a-4a0f-8d75-f74882ffbf3a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:59.077812 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7174a010-8c5a-4a0f-8d75-f74882ffbf3a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:59.085663 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7174a010-8c5a-4a0f-8d75-f74882ffbf3a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:59.086160 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7174a010-8c5a-4a0f-8d75-f74882ffbf3a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:59.087121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7174a010-8c5a-4a0f-8d75-f74882ffbf3a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:59.088012 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7174a010-8c5a-4a0f-8d75-f74882ffbf3a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:28:59.088498 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7174a010-8c5a-4a0f-8d75-f74882ffbf3a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:28:59.088931 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7174a010-8c5a-4a0f-8d75-f74882ffbf3a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:28:59.237337 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7174a010-8c5a-4a0f-8d75-f74882ffbf3a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:59.246934 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7174a010-8c5a-4a0f-8d75-f74882ffbf3a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.176249 Aug 21 01:28:59.247558 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 240/486] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:59 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:28:59.249868 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3b0d0e1d-3966-42e2-9c76-3352f5e4f9ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:28:59.251077 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3b0d0e1d-3966-42e2-9c76-3352f5e4f9ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:28:59.307752 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3b0d0e1d-3966-42e2-9c76-3352f5e4f9ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:28:59.315963 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3b0d0e1d-3966-42e2-9c76-3352f5e4f9ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1088 microversion: 2.1 time: 0.071055 Aug 21 01:28:59.316555 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 247/487] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:28:59 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1088 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:00.188795 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-471ab9a1-0a65-468d-8c2b-698ed822cb86 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:29:00.191616 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-471ab9a1-0a65-468d-8c2b-698ed822cb86 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:00.199462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-471ab9a1-0a65-468d-8c2b-698ed822cb86 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:00.200224 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-471ab9a1-0a65-468d-8c2b-698ed822cb86 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:00.200942 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-471ab9a1-0a65-468d-8c2b-698ed822cb86 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:00.202178 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-471ab9a1-0a65-468d-8c2b-698ed822cb86 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:00.202639 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-471ab9a1-0a65-468d-8c2b-698ed822cb86 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:00.203103 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-471ab9a1-0a65-468d-8c2b-698ed822cb86 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:00.229117 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-471ab9a1-0a65-468d-8c2b-698ed822cb86 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lazy-loading 'flavor' on Instance uuid eddb6b3f-8ca0-4f71-b371-19c6e65dd095 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:29:00.270262 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e060c545-08ae-46ed-9b53-a8511d4ce53e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:00.270882 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e060c545-08ae-46ed-9b53-a8511d4ce53e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:00.277427 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e060c545-08ae-46ed-9b53-a8511d4ce53e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:00.277851 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e060c545-08ae-46ed-9b53-a8511d4ce53e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:00.278312 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e060c545-08ae-46ed-9b53-a8511d4ce53e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:00.279475 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e060c545-08ae-46ed-9b53-a8511d4ce53e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:00.279934 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e060c545-08ae-46ed-9b53-a8511d4ce53e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:00.281527 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e060c545-08ae-46ed-9b53-a8511d4ce53e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:00.434528 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e060c545-08ae-46ed-9b53-a8511d4ce53e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:00.443454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e060c545-08ae-46ed-9b53-a8511d4ce53e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.178075 Aug 21 01:29:00.444035 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 248/488] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:00 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:00.448452 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c18ed67f-6327-4a78-b1d1-854e001727ec tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:00.449056 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c18ed67f-6327-4a78-b1d1-854e001727ec tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:00.454842 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c18ed67f-6327-4a78-b1d1-854e001727ec tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:00.455339 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c18ed67f-6327-4a78-b1d1-854e001727ec tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:00.455777 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c18ed67f-6327-4a78-b1d1-854e001727ec tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:00.456560 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c18ed67f-6327-4a78-b1d1-854e001727ec tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:00.457018 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c18ed67f-6327-4a78-b1d1-854e001727ec tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:00.457445 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c18ed67f-6327-4a78-b1d1-854e001727ec tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:00.530070 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c18ed67f-6327-4a78-b1d1-854e001727ec tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:00.537819 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c18ed67f-6327-4a78-b1d1-854e001727ec tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1186 microversion: 2.1 time: 0.092802 Aug 21 01:29:00.538499 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 249/489] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:00 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1186 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:00.556095 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-471ab9a1-0a65-468d-8c2b-698ed822cb86 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "POST /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/action" status: 200 len: 32 microversion: 2.1 time: 0.371865 Aug 21 01:29:00.556753 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 241/490] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:29:00 2022] POST /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/action => generated 32 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 21 01:29:00.569922 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a7a06e66-589a-43a7-8923-58ef1e6768c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:00.570536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a7a06e66-589a-43a7-8923-58ef1e6768c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:00.576553 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a7a06e66-589a-43a7-8923-58ef1e6768c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:00.577045 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a7a06e66-589a-43a7-8923-58ef1e6768c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:00.577488 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a7a06e66-589a-43a7-8923-58ef1e6768c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:00.578187 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a7a06e66-589a-43a7-8923-58ef1e6768c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:00.578644 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a7a06e66-589a-43a7-8923-58ef1e6768c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:00.579068 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a7a06e66-589a-43a7-8923-58ef1e6768c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:00.721897 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a7a06e66-589a-43a7-8923-58ef1e6768c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:00.730988 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a7a06e66-589a-43a7-8923-58ef1e6768c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.164593 Aug 21 01:29:00.731458 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 250/491] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:00 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:01.468225 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a02fe4fd-5559-4d1a-87e0-e63f6ce7ad1d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:01.469244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a02fe4fd-5559-4d1a-87e0-e63f6ce7ad1d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:01.484068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a02fe4fd-5559-4d1a-87e0-e63f6ce7ad1d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:01.484607 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a02fe4fd-5559-4d1a-87e0-e63f6ce7ad1d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:01.485053 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a02fe4fd-5559-4d1a-87e0-e63f6ce7ad1d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:01.485775 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a02fe4fd-5559-4d1a-87e0-e63f6ce7ad1d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:01.486235 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a02fe4fd-5559-4d1a-87e0-e63f6ce7ad1d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:01.486666 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a02fe4fd-5559-4d1a-87e0-e63f6ce7ad1d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:01.559744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-aa309cf1-78d3-49ba-a575-6009a07e3618 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:01.560705 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-aa309cf1-78d3-49ba-a575-6009a07e3618 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:01.568591 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aa309cf1-78d3-49ba-a575-6009a07e3618 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:01.569089 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aa309cf1-78d3-49ba-a575-6009a07e3618 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:01.569571 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aa309cf1-78d3-49ba-a575-6009a07e3618 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:01.570306 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aa309cf1-78d3-49ba-a575-6009a07e3618 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:01.570765 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aa309cf1-78d3-49ba-a575-6009a07e3618 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:01.571226 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aa309cf1-78d3-49ba-a575-6009a07e3618 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:01.681931 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-aa309cf1-78d3-49ba-a575-6009a07e3618 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:01.694050 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-aa309cf1-78d3-49ba-a575-6009a07e3618 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1204 microversion: 2.1 time: 0.137638 Aug 21 01:29:01.695060 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 251/492] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:01 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1204 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:01.758087 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-930a7efb-5cec-4506-970e-735c7316d7ad tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:01.762237 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-930a7efb-5cec-4506-970e-735c7316d7ad tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:01.766787 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a02fe4fd-5559-4d1a-87e0-e63f6ce7ad1d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:01.774908 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-930a7efb-5cec-4506-970e-735c7316d7ad tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:01.775402 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-930a7efb-5cec-4506-970e-735c7316d7ad tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:01.775839 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-930a7efb-5cec-4506-970e-735c7316d7ad tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:01.776560 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-930a7efb-5cec-4506-970e-735c7316d7ad tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:01.777210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-930a7efb-5cec-4506-970e-735c7316d7ad tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:01.777637 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-930a7efb-5cec-4506-970e-735c7316d7ad tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:01.779090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a02fe4fd-5559-4d1a-87e0-e63f6ce7ad1d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.317618 Aug 21 01:29:01.781607 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 242/493] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:01 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:01.931174 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-930a7efb-5cec-4506-970e-735c7316d7ad tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:01.941851 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-930a7efb-5cec-4506-970e-735c7316d7ad tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.192025 Aug 21 01:29:01.942764 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 252/494] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:01 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:02.721397 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f1252815-d921-460b-b2df-13396c4de11e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:02.721850 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f1252815-d921-460b-b2df-13396c4de11e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:02.728574 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1252815-d921-460b-b2df-13396c4de11e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:02.729225 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1252815-d921-460b-b2df-13396c4de11e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:02.729921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1252815-d921-460b-b2df-13396c4de11e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:02.730816 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1252815-d921-460b-b2df-13396c4de11e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:02.731278 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1252815-d921-460b-b2df-13396c4de11e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:02.731696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1252815-d921-460b-b2df-13396c4de11e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:02.800832 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-0a5b13d1-d998-42ec-baad-716cbe1fb8f5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:02.802353 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-0a5b13d1-d998-42ec-baad-716cbe1fb8f5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:02.813553 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0a5b13d1-d998-42ec-baad-716cbe1fb8f5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:02.814835 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0a5b13d1-d998-42ec-baad-716cbe1fb8f5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:02.814835 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0a5b13d1-d998-42ec-baad-716cbe1fb8f5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:02.815455 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0a5b13d1-d998-42ec-baad-716cbe1fb8f5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:02.815956 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0a5b13d1-d998-42ec-baad-716cbe1fb8f5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:02.816357 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0a5b13d1-d998-42ec-baad-716cbe1fb8f5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:02.929262 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f1252815-d921-460b-b2df-13396c4de11e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:02.939880 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f1252815-d921-460b-b2df-13396c4de11e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1272 microversion: 2.1 time: 0.223996 Aug 21 01:29:02.940433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 243/495] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:02 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1272 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:02.977632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f1ba11f7-0e16-404b-8bdb-1a7412af8dce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:02.978991 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f1ba11f7-0e16-404b-8bdb-1a7412af8dce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:02.988253 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1ba11f7-0e16-404b-8bdb-1a7412af8dce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:02.989203 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1ba11f7-0e16-404b-8bdb-1a7412af8dce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:02.989866 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1ba11f7-0e16-404b-8bdb-1a7412af8dce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:02.990801 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1ba11f7-0e16-404b-8bdb-1a7412af8dce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:02.991283 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1ba11f7-0e16-404b-8bdb-1a7412af8dce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:02.991790 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f1ba11f7-0e16-404b-8bdb-1a7412af8dce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:03.030303 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-0a5b13d1-d998-42ec-baad-716cbe1fb8f5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:03.055986 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-0a5b13d1-d998-42ec-baad-716cbe1fb8f5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.259042 Aug 21 01:29:03.068292 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 253/496] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:02 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:03.183537 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f1ba11f7-0e16-404b-8bdb-1a7412af8dce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:03.200769 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f1ba11f7-0e16-404b-8bdb-1a7412af8dce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.230820 Aug 21 01:29:03.203061 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 244/497] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:02 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:03.224942 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-4d02a7c0-c660-4dab-b9b4-d653c0d01856 req-23947d68-37b9-4cd6-a9bd-75947733fee7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "119204ec-e0f8-439e-b985-b934d251684f", "tag": "bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:29:03.235018 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-4d02a7c0-c660-4dab-b9b4-d653c0d01856 req-23947d68-37b9-4cd6-a9bd-75947733fee7 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:03.235489 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-4d02a7c0-c660-4dab-b9b4-d653c0d01856 req-23947d68-37b9-4cd6-a9bd-75947733fee7 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:03.235913 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-4d02a7c0-c660-4dab-b9b4-d653c0d01856 req-23947d68-37b9-4cd6-a9bd-75947733fee7 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:03.249151 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-4d02a7c0-c660-4dab-b9b4-d653c0d01856 req-23947d68-37b9-4cd6-a9bd-75947733fee7 service nova] Creating event network-changed:bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f for instance 119204ec-e0f8-439e-b985-b934d251684f on hv-cinder-maste Aug 21 01:29:03.254411 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-4d02a7c0-c660-4dab-b9b4-d653c0d01856 req-23947d68-37b9-4cd6-a9bd-75947733fee7 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033177 Aug 21 01:29:03.255438 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 254/498] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:29:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:29:03.961560 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-cbc8b15d-cae8-4a35-803e-b7420bd9b160 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:03.962192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-cbc8b15d-cae8-4a35-803e-b7420bd9b160 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:03.964710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=6 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:03.970199 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cbc8b15d-cae8-4a35-803e-b7420bd9b160 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:03.970699 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cbc8b15d-cae8-4a35-803e-b7420bd9b160 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:03.971151 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cbc8b15d-cae8-4a35-803e-b7420bd9b160 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:03.971866 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cbc8b15d-cae8-4a35-803e-b7420bd9b160 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:03.972351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cbc8b15d-cae8-4a35-803e-b7420bd9b160 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:03.972785 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cbc8b15d-cae8-4a35-803e-b7420bd9b160 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:04.081192 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e688f00b-9261-48c6-a85b-6f89b314d3d3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:04.082696 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e688f00b-9261-48c6-a85b-6f89b314d3d3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:04.089750 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e688f00b-9261-48c6-a85b-6f89b314d3d3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:04.090245 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e688f00b-9261-48c6-a85b-6f89b314d3d3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:04.090900 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e688f00b-9261-48c6-a85b-6f89b314d3d3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:04.091776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e688f00b-9261-48c6-a85b-6f89b314d3d3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:04.092983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e688f00b-9261-48c6-a85b-6f89b314d3d3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:04.093589 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e688f00b-9261-48c6-a85b-6f89b314d3d3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:04.126122 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-cbc8b15d-cae8-4a35-803e-b7420bd9b160 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:04.134491 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-cbc8b15d-cae8-4a35-803e-b7420bd9b160 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.176814 Aug 21 01:29:04.135161 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 245/499] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:03 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:04.231824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1a44a751-7afb-48a9-8cef-add53ba54796 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:04.233010 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1a44a751-7afb-48a9-8cef-add53ba54796 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:04.241644 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1a44a751-7afb-48a9-8cef-add53ba54796 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:04.242450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1a44a751-7afb-48a9-8cef-add53ba54796 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:04.244075 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1a44a751-7afb-48a9-8cef-add53ba54796 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:04.246240 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1a44a751-7afb-48a9-8cef-add53ba54796 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:04.246975 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1a44a751-7afb-48a9-8cef-add53ba54796 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:04.247594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1a44a751-7afb-48a9-8cef-add53ba54796 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:04.250097 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e688f00b-9261-48c6-a85b-6f89b314d3d3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:04.269336 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e688f00b-9261-48c6-a85b-6f89b314d3d3 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.193050 Aug 21 01:29:04.270257 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 255/500] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:04 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:04.416950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1a44a751-7afb-48a9-8cef-add53ba54796 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:04.432492 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1a44a751-7afb-48a9-8cef-add53ba54796 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.210880 Aug 21 01:29:04.433564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 246/501] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:04 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:05.157277 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-584acdd3-3983-496b-914d-b75c0a12da95 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:05.157997 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-584acdd3-3983-496b-914d-b75c0a12da95 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:05.164669 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-584acdd3-3983-496b-914d-b75c0a12da95 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:05.165162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-584acdd3-3983-496b-914d-b75c0a12da95 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:05.165600 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-584acdd3-3983-496b-914d-b75c0a12da95 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:05.166302 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-584acdd3-3983-496b-914d-b75c0a12da95 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:05.166754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-584acdd3-3983-496b-914d-b75c0a12da95 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:05.167180 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-584acdd3-3983-496b-914d-b75c0a12da95 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:05.293504 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8021213d-6358-459d-b254-f45bdcf5b366 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:05.294122 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8021213d-6358-459d-b254-f45bdcf5b366 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:05.301534 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8021213d-6358-459d-b254-f45bdcf5b366 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:05.302027 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8021213d-6358-459d-b254-f45bdcf5b366 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:05.302469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8021213d-6358-459d-b254-f45bdcf5b366 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:05.303175 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8021213d-6358-459d-b254-f45bdcf5b366 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:05.303630 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8021213d-6358-459d-b254-f45bdcf5b366 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:05.304086 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8021213d-6358-459d-b254-f45bdcf5b366 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:05.305295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-584acdd3-3983-496b-914d-b75c0a12da95 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:05.308823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=100 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:05.313724 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-584acdd3-3983-496b-914d-b75c0a12da95 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.160823 Aug 21 01:29:05.314487 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 256/502] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:05 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:05.442436 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8021213d-6358-459d-b254-f45bdcf5b366 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:05.450410 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8021213d-6358-459d-b254-f45bdcf5b366 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.160894 Aug 21 01:29:05.450984 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 247/503] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:05 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:05.471083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4109ddb1-0bf8-4acc-a18f-acc3979a87bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:05.471860 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4109ddb1-0bf8-4acc-a18f-acc3979a87bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:05.478266 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4109ddb1-0bf8-4acc-a18f-acc3979a87bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:05.478780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4109ddb1-0bf8-4acc-a18f-acc3979a87bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:05.479208 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4109ddb1-0bf8-4acc-a18f-acc3979a87bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:05.479932 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4109ddb1-0bf8-4acc-a18f-acc3979a87bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:05.480423 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4109ddb1-0bf8-4acc-a18f-acc3979a87bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:05.480855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4109ddb1-0bf8-4acc-a18f-acc3979a87bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:05.617916 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4109ddb1-0bf8-4acc-a18f-acc3979a87bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:05.626185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4109ddb1-0bf8-4acc-a18f-acc3979a87bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.160192 Aug 21 01:29:05.626751 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 257/504] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:05 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:06.334767 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6a50cfde-a91b-4d49-bec8-15e50e833983 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:06.335406 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6a50cfde-a91b-4d49-bec8-15e50e833983 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:06.342035 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6a50cfde-a91b-4d49-bec8-15e50e833983 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:06.342561 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6a50cfde-a91b-4d49-bec8-15e50e833983 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:06.343005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6a50cfde-a91b-4d49-bec8-15e50e833983 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:06.343696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6a50cfde-a91b-4d49-bec8-15e50e833983 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:06.344181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6a50cfde-a91b-4d49-bec8-15e50e833983 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:06.344608 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6a50cfde-a91b-4d49-bec8-15e50e833983 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:06.474296 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6a50cfde-a91b-4d49-bec8-15e50e833983 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:06.475445 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-293527d0-efeb-4995-b059-b5bfd8de255c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:06.476462 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-293527d0-efeb-4995-b059-b5bfd8de255c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:06.482818 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6a50cfde-a91b-4d49-bec8-15e50e833983 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.151851 Aug 21 01:29:06.487196 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 248/505] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:06 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:06.487350 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-293527d0-efeb-4995-b059-b5bfd8de255c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:06.487350 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-293527d0-efeb-4995-b059-b5bfd8de255c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:06.487350 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-293527d0-efeb-4995-b059-b5bfd8de255c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:06.487637 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-293527d0-efeb-4995-b059-b5bfd8de255c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:06.488218 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-293527d0-efeb-4995-b059-b5bfd8de255c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:06.488844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-293527d0-efeb-4995-b059-b5bfd8de255c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:06.629013 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-293527d0-efeb-4995-b059-b5bfd8de255c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:06.637139 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-293527d0-efeb-4995-b059-b5bfd8de255c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.168229 Aug 21 01:29:06.638003 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 258/506] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:06 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:06.646955 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-056a59f4-2410-491e-9136-9d6c4738b9ae tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:06.647565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-056a59f4-2410-491e-9136-9d6c4738b9ae tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:06.653686 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-056a59f4-2410-491e-9136-9d6c4738b9ae tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:06.654163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-056a59f4-2410-491e-9136-9d6c4738b9ae tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:06.654597 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-056a59f4-2410-491e-9136-9d6c4738b9ae tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:06.655779 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-056a59f4-2410-491e-9136-9d6c4738b9ae tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:06.656251 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-056a59f4-2410-491e-9136-9d6c4738b9ae tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:06.656675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-056a59f4-2410-491e-9136-9d6c4738b9ae tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:06.793838 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-056a59f4-2410-491e-9136-9d6c4738b9ae tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:06.802463 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-056a59f4-2410-491e-9136-9d6c4738b9ae tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.159271 Aug 21 01:29:06.803016 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 249/507] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:06 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:07.504234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4227e55d-d200-4733-bcbb-d9b1a2e4f23f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:07.504844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4227e55d-d200-4733-bcbb-d9b1a2e4f23f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:07.511146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4227e55d-d200-4733-bcbb-d9b1a2e4f23f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:07.511623 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4227e55d-d200-4733-bcbb-d9b1a2e4f23f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:07.512057 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4227e55d-d200-4733-bcbb-d9b1a2e4f23f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:07.512770 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4227e55d-d200-4733-bcbb-d9b1a2e4f23f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:07.513219 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4227e55d-d200-4733-bcbb-d9b1a2e4f23f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:07.513635 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4227e55d-d200-4733-bcbb-d9b1a2e4f23f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:07.647354 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4227e55d-d200-4733-bcbb-d9b1a2e4f23f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:07.655512 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4227e55d-d200-4733-bcbb-d9b1a2e4f23f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.155083 Aug 21 01:29:07.656195 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 259/508] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:07 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:07.658830 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-02941d84-3628-4dbd-b84b-55f67fc8f2e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:07.659523 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-02941d84-3628-4dbd-b84b-55f67fc8f2e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:07.662024 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:07.662973 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=6 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:07.667663 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-02941d84-3628-4dbd-b84b-55f67fc8f2e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:07.668187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-02941d84-3628-4dbd-b84b-55f67fc8f2e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:07.668637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-02941d84-3628-4dbd-b84b-55f67fc8f2e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:07.669338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-02941d84-3628-4dbd-b84b-55f67fc8f2e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:07.669795 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-02941d84-3628-4dbd-b84b-55f67fc8f2e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:07.670223 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-02941d84-3628-4dbd-b84b-55f67fc8f2e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:07.797763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-02941d84-3628-4dbd-b84b-55f67fc8f2e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:07.805434 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-02941d84-3628-4dbd-b84b-55f67fc8f2e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.151342 Aug 21 01:29:07.805993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 250/509] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:07 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:07.822538 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-32adf39a-5b6b-414b-a84a-48b632d44f7e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:07.825051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-32adf39a-5b6b-414b-a84a-48b632d44f7e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:07.829183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-32adf39a-5b6b-414b-a84a-48b632d44f7e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:07.829663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-32adf39a-5b6b-414b-a84a-48b632d44f7e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:07.830165 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-32adf39a-5b6b-414b-a84a-48b632d44f7e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:07.830848 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-32adf39a-5b6b-414b-a84a-48b632d44f7e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:07.831296 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-32adf39a-5b6b-414b-a84a-48b632d44f7e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:07.831714 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-32adf39a-5b6b-414b-a84a-48b632d44f7e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:07.960858 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-32adf39a-5b6b-414b-a84a-48b632d44f7e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:07.968959 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-32adf39a-5b6b-414b-a84a-48b632d44f7e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.150220 Aug 21 01:29:07.969522 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 260/510] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:07 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:08.676341 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2f689c53-0983-4848-8c57-f8c639ce11dd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:08.676929 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2f689c53-0983-4848-8c57-f8c639ce11dd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:08.683474 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2f689c53-0983-4848-8c57-f8c639ce11dd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:08.683951 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2f689c53-0983-4848-8c57-f8c639ce11dd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:08.684418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2f689c53-0983-4848-8c57-f8c639ce11dd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:08.685104 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2f689c53-0983-4848-8c57-f8c639ce11dd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:08.685553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2f689c53-0983-4848-8c57-f8c639ce11dd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:08.685969 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2f689c53-0983-4848-8c57-f8c639ce11dd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:08.827646 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6c29f85e-a74e-451e-a12a-243d8a3cb205 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:08.828291 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6c29f85e-a74e-451e-a12a-243d8a3cb205 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:08.835225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6c29f85e-a74e-451e-a12a-243d8a3cb205 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:08.835729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6c29f85e-a74e-451e-a12a-243d8a3cb205 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:08.836202 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6c29f85e-a74e-451e-a12a-243d8a3cb205 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:08.836914 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6c29f85e-a74e-451e-a12a-243d8a3cb205 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:08.837372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6c29f85e-a74e-451e-a12a-243d8a3cb205 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:08.837797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6c29f85e-a74e-451e-a12a-243d8a3cb205 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:09.111313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2f689c53-0983-4848-8c57-f8c639ce11dd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:09.120664 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2f689c53-0983-4848-8c57-f8c639ce11dd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.448087 Aug 21 01:29:09.121414 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 251/511] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:08 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:09.126840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7fcba5ba-e28f-41ef-9410-b61f839490de tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:09.127457 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7fcba5ba-e28f-41ef-9410-b61f839490de tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:09.133606 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7fcba5ba-e28f-41ef-9410-b61f839490de tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:09.134103 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7fcba5ba-e28f-41ef-9410-b61f839490de tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:09.134544 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7fcba5ba-e28f-41ef-9410-b61f839490de tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:09.135388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7fcba5ba-e28f-41ef-9410-b61f839490de tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:09.135851 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7fcba5ba-e28f-41ef-9410-b61f839490de tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:09.136517 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7fcba5ba-e28f-41ef-9410-b61f839490de tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:09.150607 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6c29f85e-a74e-451e-a12a-243d8a3cb205 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:09.156634 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6c29f85e-a74e-451e-a12a-243d8a3cb205 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1679 microversion: 2.1 time: 0.333350 Aug 21 01:29:09.157202 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 261/512] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:08 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1679 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:09.281738 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7fcba5ba-e28f-41ef-9410-b61f839490de tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:09.285272 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:09.290435 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7fcba5ba-e28f-41ef-9410-b61f839490de tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.167989 Aug 21 01:29:09.291012 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 252/513] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:09 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:10.142889 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-53441cdc-2ad9-4b41-8f62-c8a3846a856b tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:10.143480 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-53441cdc-2ad9-4b41-8f62-c8a3846a856b tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:10.145337 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=4 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:10.150682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53441cdc-2ad9-4b41-8f62-c8a3846a856b tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:10.151161 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53441cdc-2ad9-4b41-8f62-c8a3846a856b tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:10.151589 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53441cdc-2ad9-4b41-8f62-c8a3846a856b tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:10.152397 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53441cdc-2ad9-4b41-8f62-c8a3846a856b tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:10.152841 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53441cdc-2ad9-4b41-8f62-c8a3846a856b tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:10.153265 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-53441cdc-2ad9-4b41-8f62-c8a3846a856b tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:10.177347 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fd454f02-4e5b-4a8d-a3e1-9c01dd9b3904 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:10.177986 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fd454f02-4e5b-4a8d-a3e1-9c01dd9b3904 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:10.184843 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd454f02-4e5b-4a8d-a3e1-9c01dd9b3904 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:10.185332 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd454f02-4e5b-4a8d-a3e1-9c01dd9b3904 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:10.185776 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd454f02-4e5b-4a8d-a3e1-9c01dd9b3904 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:10.186466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd454f02-4e5b-4a8d-a3e1-9c01dd9b3904 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:10.186922 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd454f02-4e5b-4a8d-a3e1-9c01dd9b3904 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:10.187602 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd454f02-4e5b-4a8d-a3e1-9c01dd9b3904 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:10.340433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-53441cdc-2ad9-4b41-8f62-c8a3846a856b tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:10.350882 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-53441cdc-2ad9-4b41-8f62-c8a3846a856b tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.212288 Aug 21 01:29:10.351488 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 262/514] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:10 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:10.356962 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9289dc41-7c87-451e-9a33-a1abc180e5d5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:10.357812 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9289dc41-7c87-451e-9a33-a1abc180e5d5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:10.363882 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9289dc41-7c87-451e-9a33-a1abc180e5d5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:10.364620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9289dc41-7c87-451e-9a33-a1abc180e5d5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:10.365051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9289dc41-7c87-451e-9a33-a1abc180e5d5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:10.365798 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9289dc41-7c87-451e-9a33-a1abc180e5d5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:10.366254 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9289dc41-7c87-451e-9a33-a1abc180e5d5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:10.366680 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9289dc41-7c87-451e-9a33-a1abc180e5d5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:10.369587 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fd454f02-4e5b-4a8d-a3e1-9c01dd9b3904 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:10.378482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fd454f02-4e5b-4a8d-a3e1-9c01dd9b3904 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1656 microversion: 2.1 time: 0.205332 Aug 21 01:29:10.379102 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 253/515] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:10 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1656 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:10.402025 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b4a622fe-7bc0-462b-af0f-f31807d4faa0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:29:10.409333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b4a622fe-7bc0-462b-af0f-f31807d4faa0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:10.409915 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b4a622fe-7bc0-462b-af0f-f31807d4faa0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:10.410427 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b4a622fe-7bc0-462b-af0f-f31807d4faa0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:10.432778 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-b4a622fe-7bc0-462b-af0f-f31807d4faa0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Going to try to start instance {{(pid=121276) start /opt/stack/nova/nova/compute/api.py:2772}} Aug 21 01:29:10.501981 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9289dc41-7c87-451e-9a33-a1abc180e5d5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:10.510428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9289dc41-7c87-451e-9a33-a1abc180e5d5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.157015 Aug 21 01:29:10.516185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 263/516] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:10 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:10.601192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b4a622fe-7bc0-462b-af0f-f31807d4faa0 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "POST /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/action" status: 202 len: 0 microversion: 2.1 time: 0.204329 Aug 21 01:29:10.601721 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 254/517] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:29:10 2022] POST /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/action => generated 0 bytes in 206 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:29:10.609986 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8c63df98-aa96-4dea-bfb8-55d49c93a3b5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:10.610598 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8c63df98-aa96-4dea-bfb8-55d49c93a3b5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:10.616603 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c63df98-aa96-4dea-bfb8-55d49c93a3b5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:10.617124 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c63df98-aa96-4dea-bfb8-55d49c93a3b5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:10.617566 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c63df98-aa96-4dea-bfb8-55d49c93a3b5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:10.618256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c63df98-aa96-4dea-bfb8-55d49c93a3b5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:10.618710 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c63df98-aa96-4dea-bfb8-55d49c93a3b5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:10.619133 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c63df98-aa96-4dea-bfb8-55d49c93a3b5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:10.763703 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8c63df98-aa96-4dea-bfb8-55d49c93a3b5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:10.772031 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8c63df98-aa96-4dea-bfb8-55d49c93a3b5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1665 microversion: 2.1 time: 0.165894 Aug 21 01:29:10.772623 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 264/518] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:10 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1665 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:11.372899 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-47eb1727-8093-4517-8695-3cf99d3f0df2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:11.373595 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-47eb1727-8093-4517-8695-3cf99d3f0df2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:11.379916 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-47eb1727-8093-4517-8695-3cf99d3f0df2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:11.380438 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-47eb1727-8093-4517-8695-3cf99d3f0df2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:11.380880 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-47eb1727-8093-4517-8695-3cf99d3f0df2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:11.381574 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-47eb1727-8093-4517-8695-3cf99d3f0df2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:11.382028 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-47eb1727-8093-4517-8695-3cf99d3f0df2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:11.382453 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-47eb1727-8093-4517-8695-3cf99d3f0df2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:11.518649 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-47eb1727-8093-4517-8695-3cf99d3f0df2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:11.526652 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-47eb1727-8093-4517-8695-3cf99d3f0df2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.158315 Aug 21 01:29:11.528185 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 255/519] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:11 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:11.532956 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8db19239-8e5b-495b-a0fe-4268c8663f06 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:11.533568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8db19239-8e5b-495b-a0fe-4268c8663f06 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:11.540538 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8db19239-8e5b-495b-a0fe-4268c8663f06 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:11.541038 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8db19239-8e5b-495b-a0fe-4268c8663f06 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:11.541480 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8db19239-8e5b-495b-a0fe-4268c8663f06 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:11.542184 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8db19239-8e5b-495b-a0fe-4268c8663f06 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:11.542645 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8db19239-8e5b-495b-a0fe-4268c8663f06 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:11.543071 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8db19239-8e5b-495b-a0fe-4268c8663f06 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:11.678003 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8db19239-8e5b-495b-a0fe-4268c8663f06 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:11.686471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8db19239-8e5b-495b-a0fe-4268c8663f06 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.157871 Aug 21 01:29:11.687053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 265/520] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:11 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:11.793218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-eb4d9af7-4c5d-40c8-ada5-c30d7b097bdf tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:11.793838 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-eb4d9af7-4c5d-40c8-ada5-c30d7b097bdf tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:11.800249 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb4d9af7-4c5d-40c8-ada5-c30d7b097bdf tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:11.800758 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb4d9af7-4c5d-40c8-ada5-c30d7b097bdf tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:11.801221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb4d9af7-4c5d-40c8-ada5-c30d7b097bdf tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:11.801927 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb4d9af7-4c5d-40c8-ada5-c30d7b097bdf tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:11.802383 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb4d9af7-4c5d-40c8-ada5-c30d7b097bdf tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:11.802813 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb4d9af7-4c5d-40c8-ada5-c30d7b097bdf tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:11.948543 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-eb4d9af7-4c5d-40c8-ada5-c30d7b097bdf tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:11.957132 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-eb4d9af7-4c5d-40c8-ada5-c30d7b097bdf tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1665 microversion: 2.1 time: 0.167948 Aug 21 01:29:11.957725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 256/521] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:11 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1665 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:12.548493 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f7e46428-32b1-4bd4-8f46-83abfe2f0702 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:12.549015 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f7e46428-32b1-4bd4-8f46-83abfe2f0702 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:12.555398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7e46428-32b1-4bd4-8f46-83abfe2f0702 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:12.555886 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7e46428-32b1-4bd4-8f46-83abfe2f0702 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:12.556356 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7e46428-32b1-4bd4-8f46-83abfe2f0702 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:12.557044 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7e46428-32b1-4bd4-8f46-83abfe2f0702 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:12.557499 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7e46428-32b1-4bd4-8f46-83abfe2f0702 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:12.557923 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7e46428-32b1-4bd4-8f46-83abfe2f0702 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:12.688272 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f7e46428-32b1-4bd4-8f46-83abfe2f0702 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:12.695978 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f7e46428-32b1-4bd4-8f46-83abfe2f0702 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.151981 Aug 21 01:29:12.696586 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 266/522] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:12 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:12.706704 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a5e2de5b-7520-459e-a2ca-225c29f8b664 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:12.707314 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a5e2de5b-7520-459e-a2ca-225c29f8b664 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:12.714192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5e2de5b-7520-459e-a2ca-225c29f8b664 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:12.714683 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5e2de5b-7520-459e-a2ca-225c29f8b664 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:12.715128 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5e2de5b-7520-459e-a2ca-225c29f8b664 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:12.715827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5e2de5b-7520-459e-a2ca-225c29f8b664 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:12.716316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5e2de5b-7520-459e-a2ca-225c29f8b664 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:12.716741 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5e2de5b-7520-459e-a2ca-225c29f8b664 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:12.850746 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a5e2de5b-7520-459e-a2ca-225c29f8b664 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:12.859739 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a5e2de5b-7520-459e-a2ca-225c29f8b664 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.156258 Aug 21 01:29:12.860322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 257/523] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:12 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:12.978656 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4b602dac-a878-401e-a716-1f271925a153 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:12.979266 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4b602dac-a878-401e-a716-1f271925a153 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:12.985837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4b602dac-a878-401e-a716-1f271925a153 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:12.986318 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4b602dac-a878-401e-a716-1f271925a153 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:12.986750 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4b602dac-a878-401e-a716-1f271925a153 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:12.987427 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4b602dac-a878-401e-a716-1f271925a153 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:12.987879 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4b602dac-a878-401e-a716-1f271925a153 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:12.988329 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4b602dac-a878-401e-a716-1f271925a153 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:13.121220 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4b602dac-a878-401e-a716-1f271925a153 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:13.130357 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4b602dac-a878-401e-a716-1f271925a153 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1665 microversion: 2.1 time: 0.155494 Aug 21 01:29:13.130930 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 267/524] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:12 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1665 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:13.716886 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-885a6445-93d9-4a81-b1bb-20b4fd0c8790 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:13.717501 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-885a6445-93d9-4a81-b1bb-20b4fd0c8790 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:13.723918 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-885a6445-93d9-4a81-b1bb-20b4fd0c8790 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:13.724449 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-885a6445-93d9-4a81-b1bb-20b4fd0c8790 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:13.724884 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-885a6445-93d9-4a81-b1bb-20b4fd0c8790 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:13.725565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-885a6445-93d9-4a81-b1bb-20b4fd0c8790 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:13.726021 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-885a6445-93d9-4a81-b1bb-20b4fd0c8790 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:13.726444 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-885a6445-93d9-4a81-b1bb-20b4fd0c8790 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:13.855657 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-885a6445-93d9-4a81-b1bb-20b4fd0c8790 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:13.863811 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-885a6445-93d9-4a81-b1bb-20b4fd0c8790 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.150853 Aug 21 01:29:13.864405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 258/525] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:13 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:13.881954 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8df11750-3116-45a3-a17e-5b97c2899edd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:13.882601 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8df11750-3116-45a3-a17e-5b97c2899edd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:13.888630 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8df11750-3116-45a3-a17e-5b97c2899edd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:13.889116 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8df11750-3116-45a3-a17e-5b97c2899edd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:13.889555 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8df11750-3116-45a3-a17e-5b97c2899edd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:13.890349 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8df11750-3116-45a3-a17e-5b97c2899edd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:13.890815 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8df11750-3116-45a3-a17e-5b97c2899edd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:13.891246 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8df11750-3116-45a3-a17e-5b97c2899edd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:14.027319 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8df11750-3116-45a3-a17e-5b97c2899edd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:14.036282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8df11750-3116-45a3-a17e-5b97c2899edd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.158328 Aug 21 01:29:14.036877 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 268/526] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:13 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:14.150711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6c3f1af8-c347-4fa1-b095-1b08f7488811 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:14.151350 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6c3f1af8-c347-4fa1-b095-1b08f7488811 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:14.158191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c3f1af8-c347-4fa1-b095-1b08f7488811 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:14.158679 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c3f1af8-c347-4fa1-b095-1b08f7488811 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:14.159156 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c3f1af8-c347-4fa1-b095-1b08f7488811 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:14.159886 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c3f1af8-c347-4fa1-b095-1b08f7488811 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:14.160388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c3f1af8-c347-4fa1-b095-1b08f7488811 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:14.161562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c3f1af8-c347-4fa1-b095-1b08f7488811 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:14.291458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6c3f1af8-c347-4fa1-b095-1b08f7488811 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:14.299947 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6c3f1af8-c347-4fa1-b095-1b08f7488811 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1669 microversion: 2.1 time: 0.153033 Aug 21 01:29:14.300570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 259/527] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:14 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1669 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:14.884769 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d2abb1cc-8e11-47a4-be6f-ff404f87f4b2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:14.885380 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d2abb1cc-8e11-47a4-be6f-ff404f87f4b2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:14.892325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d2abb1cc-8e11-47a4-be6f-ff404f87f4b2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:14.892819 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d2abb1cc-8e11-47a4-be6f-ff404f87f4b2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:14.893267 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d2abb1cc-8e11-47a4-be6f-ff404f87f4b2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:14.893968 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d2abb1cc-8e11-47a4-be6f-ff404f87f4b2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:14.894472 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d2abb1cc-8e11-47a4-be6f-ff404f87f4b2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:14.894903 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d2abb1cc-8e11-47a4-be6f-ff404f87f4b2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:15.030595 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d2abb1cc-8e11-47a4-be6f-ff404f87f4b2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:15.039115 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d2abb1cc-8e11-47a4-be6f-ff404f87f4b2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.158286 Aug 21 01:29:15.039704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 269/528] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:14 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:15.058512 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-64c2d16d-9dc3-4d77-95df-d985b598cb1e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:15.059328 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-64c2d16d-9dc3-4d77-95df-d985b598cb1e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:15.066265 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-64c2d16d-9dc3-4d77-95df-d985b598cb1e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:15.066788 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-64c2d16d-9dc3-4d77-95df-d985b598cb1e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:15.067222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-64c2d16d-9dc3-4d77-95df-d985b598cb1e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:15.067916 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-64c2d16d-9dc3-4d77-95df-d985b598cb1e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:15.068390 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-64c2d16d-9dc3-4d77-95df-d985b598cb1e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:15.068807 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-64c2d16d-9dc3-4d77-95df-d985b598cb1e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:15.099748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7197d61e-417e-4235-80a3-890e12498d29 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b4192168-4093-4570-b7e8-d4767e661456", "device": "/dev/sdb"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:29:15.102612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7197d61e-417e-4235-80a3-890e12498d29 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:15.108742 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7197d61e-417e-4235-80a3-890e12498d29 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:15.109802 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7197d61e-417e-4235-80a3-890e12498d29 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:15.110558 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7197d61e-417e-4235-80a3-890e12498d29 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:15.112513 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7197d61e-417e-4235-80a3-890e12498d29 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:15.112513 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7197d61e-417e-4235-80a3-890e12498d29 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:15.112513 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7197d61e-417e-4235-80a3-890e12498d29 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:15.216283 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-64c2d16d-9dc3-4d77-95df-d985b598cb1e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:15.226319 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-64c2d16d-9dc3-4d77-95df-d985b598cb1e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.171534 Aug 21 01:29:15.226905 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 260/529] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:15 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:15.933051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7197d61e-417e-4235-80a3-890e12498d29 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "POST /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.837153 Aug 21 01:29:15.933743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 270/530] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:29:15 2022] POST /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23/os-volume_attachments => generated 194 bytes in 839 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:29:16.061029 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d567e9ef-b9b8-4756-81fd-ee354bd1e5a1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:16.061569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d567e9ef-b9b8-4756-81fd-ee354bd1e5a1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:16.069032 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d567e9ef-b9b8-4756-81fd-ee354bd1e5a1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:16.069533 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d567e9ef-b9b8-4756-81fd-ee354bd1e5a1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:16.069958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d567e9ef-b9b8-4756-81fd-ee354bd1e5a1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:16.070643 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d567e9ef-b9b8-4756-81fd-ee354bd1e5a1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:16.071097 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d567e9ef-b9b8-4756-81fd-ee354bd1e5a1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:16.071521 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d567e9ef-b9b8-4756-81fd-ee354bd1e5a1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:16.217288 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d567e9ef-b9b8-4756-81fd-ee354bd1e5a1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:16.225642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d567e9ef-b9b8-4756-81fd-ee354bd1e5a1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.168539 Aug 21 01:29:16.226223 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 261/531] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:16 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:16.258679 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e07ca661-49d3-4d07-8c91-f051dbb82357 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:16.259660 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e07ca661-49d3-4d07-8c91-f051dbb82357 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:16.268982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e07ca661-49d3-4d07-8c91-f051dbb82357 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:16.269593 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e07ca661-49d3-4d07-8c91-f051dbb82357 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:16.270083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e07ca661-49d3-4d07-8c91-f051dbb82357 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:16.270912 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e07ca661-49d3-4d07-8c91-f051dbb82357 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:16.271416 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e07ca661-49d3-4d07-8c91-f051dbb82357 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:16.271887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e07ca661-49d3-4d07-8c91-f051dbb82357 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:16.405024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e07ca661-49d3-4d07-8c91-f051dbb82357 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:16.413956 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e07ca661-49d3-4d07-8c91-f051dbb82357 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.159424 Aug 21 01:29:16.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 271/532] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:16 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:16.834807 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-726f7606-303e-495c-a0a4-cf757fd11b17 req-2c5a478d-3795-4c12-8856-9c3d62a6c65b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "119204ec-e0f8-439e-b985-b934d251684f", "name": "network-vif-plugged", "status": "completed", "tag": "bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:29:16.845434 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-726f7606-303e-495c-a0a4-cf757fd11b17 req-2c5a478d-3795-4c12-8856-9c3d62a6c65b service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:16.845924 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-726f7606-303e-495c-a0a4-cf757fd11b17 req-2c5a478d-3795-4c12-8856-9c3d62a6c65b service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:16.846355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-726f7606-303e-495c-a0a4-cf757fd11b17 req-2c5a478d-3795-4c12-8856-9c3d62a6c65b service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:16.858033 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-726f7606-303e-495c-a0a4-cf757fd11b17 req-2c5a478d-3795-4c12-8856-9c3d62a6c65b service nova] Creating event network-vif-plugged:bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f for instance 119204ec-e0f8-439e-b985-b934d251684f on hv-cinder-maste Aug 21 01:29:16.863678 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-726f7606-303e-495c-a0a4-cf757fd11b17 req-2c5a478d-3795-4c12-8856-9c3d62a6c65b service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032003 Aug 21 01:29:16.863678 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 262/533] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:29:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:29:17.249370 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-40b8db30-fc46-4749-bb5e-f4101d9c7b76 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:17.250693 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-40b8db30-fc46-4749-bb5e-f4101d9c7b76 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:17.257071 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-40b8db30-fc46-4749-bb5e-f4101d9c7b76 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:17.257682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-40b8db30-fc46-4749-bb5e-f4101d9c7b76 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:17.258128 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-40b8db30-fc46-4749-bb5e-f4101d9c7b76 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:17.258838 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-40b8db30-fc46-4749-bb5e-f4101d9c7b76 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:17.259295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-40b8db30-fc46-4749-bb5e-f4101d9c7b76 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:17.259737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-40b8db30-fc46-4749-bb5e-f4101d9c7b76 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:17.264744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:17.424669 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-40b8db30-fc46-4749-bb5e-f4101d9c7b76 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:17.435884 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b23908ec-0975-4a82-835a-2cf39b93e9e0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:17.436817 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-40b8db30-fc46-4749-bb5e-f4101d9c7b76 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.191224 Aug 21 01:29:17.438394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-b23908ec-0975-4a82-835a-2cf39b93e9e0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:17.446279 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b23908ec-0975-4a82-835a-2cf39b93e9e0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:17.447224 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b23908ec-0975-4a82-835a-2cf39b93e9e0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:17.447838 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b23908ec-0975-4a82-835a-2cf39b93e9e0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:17.448782 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b23908ec-0975-4a82-835a-2cf39b93e9e0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:17.449244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b23908ec-0975-4a82-835a-2cf39b93e9e0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:17.449675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b23908ec-0975-4a82-835a-2cf39b93e9e0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:17.452738 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 272/534] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:17 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:17.631407 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b23908ec-0975-4a82-835a-2cf39b93e9e0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:17.645750 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b23908ec-0975-4a82-835a-2cf39b93e9e0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.213024 Aug 21 01:29:17.647535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 263/535] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:17 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:18.458934 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ca511961-4185-415f-b93b-d186b24e7bbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:18.459554 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-ca511961-4185-415f-b93b-d186b24e7bbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:18.462217 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=86 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:18.467272 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ca511961-4185-415f-b93b-d186b24e7bbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:18.467766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ca511961-4185-415f-b93b-d186b24e7bbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:18.468231 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ca511961-4185-415f-b93b-d186b24e7bbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:18.468934 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ca511961-4185-415f-b93b-d186b24e7bbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:18.469392 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ca511961-4185-415f-b93b-d186b24e7bbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:18.469819 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ca511961-4185-415f-b93b-d186b24e7bbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:18.626263 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ca511961-4185-415f-b93b-d186b24e7bbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:18.634455 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ca511961-4185-415f-b93b-d186b24e7bbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.179395 Aug 21 01:29:18.635114 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 273/536] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:18 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:18.673593 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9083dde7-8cae-4309-a4f3-d147940536b7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:18.674653 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9083dde7-8cae-4309-a4f3-d147940536b7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:18.691111 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9083dde7-8cae-4309-a4f3-d147940536b7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:18.692001 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9083dde7-8cae-4309-a4f3-d147940536b7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:18.692843 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9083dde7-8cae-4309-a4f3-d147940536b7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:18.694016 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9083dde7-8cae-4309-a4f3-d147940536b7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:18.694550 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9083dde7-8cae-4309-a4f3-d147940536b7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:18.695050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9083dde7-8cae-4309-a4f3-d147940536b7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:18.847992 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9083dde7-8cae-4309-a4f3-d147940536b7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:18.856762 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9083dde7-8cae-4309-a4f3-d147940536b7 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.190054 Aug 21 01:29:18.857338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 264/537] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:18 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:19.655326 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-63075703-b2a4-4e14-865d-ff360877462f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:19.657263 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-63075703-b2a4-4e14-865d-ff360877462f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:19.663817 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63075703-b2a4-4e14-865d-ff360877462f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:19.664340 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63075703-b2a4-4e14-865d-ff360877462f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:19.666055 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63075703-b2a4-4e14-865d-ff360877462f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:19.666055 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63075703-b2a4-4e14-865d-ff360877462f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:19.666055 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63075703-b2a4-4e14-865d-ff360877462f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:19.666566 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63075703-b2a4-4e14-865d-ff360877462f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:19.866499 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-63075703-b2a4-4e14-865d-ff360877462f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:19.874534 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-63075703-b2a4-4e14-865d-ff360877462f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1260 microversion: 2.1 time: 0.223363 Aug 21 01:29:19.875626 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 274/538] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:19 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1260 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:19.881654 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8a4d7771-d124-4814-8bc2-2d4e0d99005b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:19.882334 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8a4d7771-d124-4814-8bc2-2d4e0d99005b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:19.888922 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a4d7771-d124-4814-8bc2-2d4e0d99005b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:19.889414 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a4d7771-d124-4814-8bc2-2d4e0d99005b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:19.889922 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a4d7771-d124-4814-8bc2-2d4e0d99005b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:19.890621 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a4d7771-d124-4814-8bc2-2d4e0d99005b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:19.891120 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a4d7771-d124-4814-8bc2-2d4e0d99005b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:19.891531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a4d7771-d124-4814-8bc2-2d4e0d99005b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:20.035695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8a4d7771-d124-4814-8bc2-2d4e0d99005b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:20.045586 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8a4d7771-d124-4814-8bc2-2d4e0d99005b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.167822 Aug 21 01:29:20.046153 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 265/539] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:19 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:20.896281 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7d161093-1aff-46c6-97f7-6fe11d0cfb00 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:20.897186 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7d161093-1aff-46c6-97f7-6fe11d0cfb00 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:20.904337 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7d161093-1aff-46c6-97f7-6fe11d0cfb00 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:20.904826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7d161093-1aff-46c6-97f7-6fe11d0cfb00 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:20.905254 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7d161093-1aff-46c6-97f7-6fe11d0cfb00 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:20.905948 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7d161093-1aff-46c6-97f7-6fe11d0cfb00 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:20.907015 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7d161093-1aff-46c6-97f7-6fe11d0cfb00 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:20.907481 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7d161093-1aff-46c6-97f7-6fe11d0cfb00 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:21.055294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7d161093-1aff-46c6-97f7-6fe11d0cfb00 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:21.064479 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7d161093-1aff-46c6-97f7-6fe11d0cfb00 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1444 microversion: 2.1 time: 0.172432 Aug 21 01:29:21.065056 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 275/540] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:20 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1444 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:21.068504 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-30484983-723a-44d3-80c0-16f92c6e5bd9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:21.069415 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-30484983-723a-44d3-80c0-16f92c6e5bd9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:21.082047 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30484983-723a-44d3-80c0-16f92c6e5bd9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:21.082912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30484983-723a-44d3-80c0-16f92c6e5bd9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:21.083687 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30484983-723a-44d3-80c0-16f92c6e5bd9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:21.084961 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30484983-723a-44d3-80c0-16f92c6e5bd9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:21.085732 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30484983-723a-44d3-80c0-16f92c6e5bd9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:21.086119 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7941ddd5-b257-40a0-87ee-2a806baf4659 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:21.086232 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30484983-723a-44d3-80c0-16f92c6e5bd9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:21.086870 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7941ddd5-b257-40a0-87ee-2a806baf4659 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:21.093067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7941ddd5-b257-40a0-87ee-2a806baf4659 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:21.093550 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7941ddd5-b257-40a0-87ee-2a806baf4659 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:21.094010 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7941ddd5-b257-40a0-87ee-2a806baf4659 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:21.094710 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7941ddd5-b257-40a0-87ee-2a806baf4659 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:21.095156 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7941ddd5-b257-40a0-87ee-2a806baf4659 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:21.095579 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7941ddd5-b257-40a0-87ee-2a806baf4659 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:21.299059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-30484983-723a-44d3-80c0-16f92c6e5bd9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:21.300227 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7941ddd5-b257-40a0-87ee-2a806baf4659 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:21.310096 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7941ddd5-b257-40a0-87ee-2a806baf4659 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1444 microversion: 2.1 time: 0.227914 Aug 21 01:29:21.317959 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-30484983-723a-44d3-80c0-16f92c6e5bd9 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1734 microversion: 2.1 time: 0.254293 Aug 21 01:29:21.318847 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 276/541] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:21 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1444 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:21.319040 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 266/542] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:21 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1734 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:22.361413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-cf9fcfae-5ec8-4aea-adaa-3edf810120ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:22.362745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-cf9fcfae-5ec8-4aea-adaa-3edf810120ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:22.370851 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cf9fcfae-5ec8-4aea-adaa-3edf810120ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:22.371482 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cf9fcfae-5ec8-4aea-adaa-3edf810120ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:22.372191 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cf9fcfae-5ec8-4aea-adaa-3edf810120ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:22.373061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cf9fcfae-5ec8-4aea-adaa-3edf810120ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:22.373639 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cf9fcfae-5ec8-4aea-adaa-3edf810120ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:22.373939 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cf9fcfae-5ec8-4aea-adaa-3edf810120ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:22.856058 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-cf9fcfae-5ec8-4aea-adaa-3edf810120ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:22.865016 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-cf9fcfae-5ec8-4aea-adaa-3edf810120ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1714 microversion: 2.1 time: 0.511331 Aug 21 01:29:22.867822 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 277/543] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:22 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1714 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:22.995198 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-da89cd6a-520e-45d9-9b15-99c690828e3d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:22.995867 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-da89cd6a-520e-45d9-9b15-99c690828e3d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:23.004320 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-da89cd6a-520e-45d9-9b15-99c690828e3d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:23.005062 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-da89cd6a-520e-45d9-9b15-99c690828e3d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:23.005564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-da89cd6a-520e-45d9-9b15-99c690828e3d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:23.006316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-da89cd6a-520e-45d9-9b15-99c690828e3d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:23.006962 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-da89cd6a-520e-45d9-9b15-99c690828e3d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:23.007182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-da89cd6a-520e-45d9-9b15-99c690828e3d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:23.099425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-da89cd6a-520e-45d9-9b15-99c690828e3d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] HTTP exception thrown: Cannot 'detach_volume' instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 while it is in vm_state rescued Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-da89cd6a-520e-45d9-9b15-99c690828e3d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Request method failure captured: Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: DELETE /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments/2fb97ee5-b984-45f9-8ae3-f2209e4b20e3 HTTP/1.1 Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYl2grkk6r6DOR4WcdCDQaZ1LKSwt5ty8EvPKEsirHl25Bbz12fukVsuTZFRPo5OnAmJDHzJfldGu7OE-fBThLL0sUCHPYzsZbzZPojBGa0s89nUPxFfxXEogZCrYm2ZzOthYXvM-jM7ah9S1dz6CN84xJGSagaAUIMSBiUILa4u8mo Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 9774c354b8b048768f2071acfb48a4bf Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:29:23.101068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:29:23.103277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:29:23.103277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:29:23.103277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 9774c354b8b048768f2071acfb48a4bf Aug 21 01:29:23.103277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:29:23.103277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-ServerRescueNegativeTestJSON-1204886246-project Aug 21 01:29:23.103277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:29:23.103277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:29:23.103277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: d84b6d05decb47c7b55ac782d5839c1a Aug 21 01:29:23.103277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1204886246-project Aug 21 01:29:23.103277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:29:23.103277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Cannot 'detach_volume' instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 while it is in vm_state rescued Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'server_id': 'eddb6b3f-8ca0-4f71-b371-19c6e65dd095', 'id': '2fb97ee5-b984-45f9-8ae3-f2209e4b20e3'} Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 in vm_state rescued. Cannot detach_volume while the instance is in this state. Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:29:23.104450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 while it is in vm_state rescued Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 while it is in vm_state rescued Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:29:23.107760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-da89cd6a-520e-45d9-9b15-99c690828e3d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Returning 409 to user: Cannot 'detach_volume' instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 while it is in vm_state rescued {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:29:23.111383 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-da89cd6a-520e-45d9-9b15-99c690828e3d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "DELETE /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments/2fb97ee5-b984-45f9-8ae3-f2209e4b20e3" status: 409 len: 152 microversion: 2.1 time: 0.111347 Aug 21 01:29:23.111383 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 267/544] 10.222.0.47 () {58 vars in 1416 bytes} [Sun Aug 21 01:29:22 2022] DELETE /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments/2fb97ee5-b984-45f9-8ae3-f2209e4b20e3 => generated 152 bytes in 114 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Aug 21 01:29:23.111816 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f8405e72-dedd-400c-8698-01047301a239 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:29:23.112315 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f8405e72-dedd-400c-8698-01047301a239 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:23.119052 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8405e72-dedd-400c-8698-01047301a239 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:23.119552 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8405e72-dedd-400c-8698-01047301a239 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:23.122070 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8405e72-dedd-400c-8698-01047301a239 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:23.122070 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8405e72-dedd-400c-8698-01047301a239 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:23.122070 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8405e72-dedd-400c-8698-01047301a239 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:23.122070 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8405e72-dedd-400c-8698-01047301a239 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:23.363666 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f8405e72-dedd-400c-8698-01047301a239 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "POST /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/action" status: 202 len: 0 microversion: 2.1 time: 0.256008 Aug 21 01:29:23.364525 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 278/545] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:29:23 2022] POST /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/action => generated 0 bytes in 258 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 21 01:29:23.372903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6bd3849f-5ccd-4343-b602-9c588e577988 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:23.373527 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6bd3849f-5ccd-4343-b602-9c588e577988 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:23.382601 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6bd3849f-5ccd-4343-b602-9c588e577988 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:23.391551 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6bd3849f-5ccd-4343-b602-9c588e577988 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:23.391551 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6bd3849f-5ccd-4343-b602-9c588e577988 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:23.391551 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6bd3849f-5ccd-4343-b602-9c588e577988 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:23.391551 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6bd3849f-5ccd-4343-b602-9c588e577988 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:23.392328 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6bd3849f-5ccd-4343-b602-9c588e577988 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:23.873163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6bd3849f-5ccd-4343-b602-9c588e577988 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:23.893305 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6bd3849f-5ccd-4343-b602-9c588e577988 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1722 microversion: 2.1 time: 0.523708 Aug 21 01:29:23.906351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 268/546] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:23 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1722 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:24.248695 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-b3453d02-c6db-48b7-9ab5-6deca9e3ce21 req-ef64340e-72b8-41b4-bcee-60290d18ef7b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "119204ec-e0f8-439e-b985-b934d251684f", "tag": "bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:29:24.260042 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-b3453d02-c6db-48b7-9ab5-6deca9e3ce21 req-ef64340e-72b8-41b4-bcee-60290d18ef7b service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:24.260573 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-b3453d02-c6db-48b7-9ab5-6deca9e3ce21 req-ef64340e-72b8-41b4-bcee-60290d18ef7b service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:24.261014 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-b3453d02-c6db-48b7-9ab5-6deca9e3ce21 req-ef64340e-72b8-41b4-bcee-60290d18ef7b service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:24.273385 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-b3453d02-c6db-48b7-9ab5-6deca9e3ce21 req-ef64340e-72b8-41b4-bcee-60290d18ef7b service nova] Creating event network-changed:bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f for instance 119204ec-e0f8-439e-b985-b934d251684f on hv-cinder-maste Aug 21 01:29:24.278482 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-b3453d02-c6db-48b7-9ab5-6deca9e3ce21 req-ef64340e-72b8-41b4-bcee-60290d18ef7b service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033788 Aug 21 01:29:24.280755 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 279/547] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:29:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:29:24.929912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2bbcf580-d34e-483c-b3bb-1eed129210f1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:24.929912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2bbcf580-d34e-483c-b3bb-1eed129210f1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:24.936553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2bbcf580-d34e-483c-b3bb-1eed129210f1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:24.937036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2bbcf580-d34e-483c-b3bb-1eed129210f1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:24.937489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2bbcf580-d34e-483c-b3bb-1eed129210f1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:24.938236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2bbcf580-d34e-483c-b3bb-1eed129210f1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:24.938689 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2bbcf580-d34e-483c-b3bb-1eed129210f1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:24.939366 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2bbcf580-d34e-483c-b3bb-1eed129210f1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:24.943607 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:25.282740 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2bbcf580-d34e-483c-b3bb-1eed129210f1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:25.302701 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2bbcf580-d34e-483c-b3bb-1eed129210f1 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1722 microversion: 2.1 time: 0.377775 Aug 21 01:29:25.304480 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 269/548] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:24 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1722 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:26.331862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-cb1f3949-f8a8-40b9-af00-afaf6f7c5919 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:26.332591 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-cb1f3949-f8a8-40b9-af00-afaf6f7c5919 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:26.339560 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb1f3949-f8a8-40b9-af00-afaf6f7c5919 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:26.340046 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb1f3949-f8a8-40b9-af00-afaf6f7c5919 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:26.340505 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb1f3949-f8a8-40b9-af00-afaf6f7c5919 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:26.341200 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb1f3949-f8a8-40b9-af00-afaf6f7c5919 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:26.341690 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb1f3949-f8a8-40b9-af00-afaf6f7c5919 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:26.342171 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cb1f3949-f8a8-40b9-af00-afaf6f7c5919 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:26.485770 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-cb1f3949-f8a8-40b9-af00-afaf6f7c5919 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:26.494574 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-cb1f3949-f8a8-40b9-af00-afaf6f7c5919 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1722 microversion: 2.1 time: 0.167035 Aug 21 01:29:26.495173 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 280/549] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:26 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1722 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:27.515749 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-dbfac0c7-05e9-4195-b641-1b98c95cd045 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:27.516398 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-dbfac0c7-05e9-4195-b641-1b98c95cd045 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:27.523141 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbfac0c7-05e9-4195-b641-1b98c95cd045 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:27.523621 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbfac0c7-05e9-4195-b641-1b98c95cd045 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:27.524116 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbfac0c7-05e9-4195-b641-1b98c95cd045 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:27.524813 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbfac0c7-05e9-4195-b641-1b98c95cd045 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:27.525498 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbfac0c7-05e9-4195-b641-1b98c95cd045 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:27.526218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dbfac0c7-05e9-4195-b641-1b98c95cd045 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:27.675282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-dbfac0c7-05e9-4195-b641-1b98c95cd045 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:27.685619 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-dbfac0c7-05e9-4195-b641-1b98c95cd045 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1722 microversion: 2.1 time: 0.173573 Aug 21 01:29:27.686202 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 270/550] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:27 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1722 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:28.707654 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-dfdb8c54-3e98-4ff0-9df2-21931ae71faa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:28.708486 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-dfdb8c54-3e98-4ff0-9df2-21931ae71faa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:28.714930 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dfdb8c54-3e98-4ff0-9df2-21931ae71faa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:28.715418 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dfdb8c54-3e98-4ff0-9df2-21931ae71faa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:28.715861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dfdb8c54-3e98-4ff0-9df2-21931ae71faa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:28.716599 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dfdb8c54-3e98-4ff0-9df2-21931ae71faa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:28.717058 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dfdb8c54-3e98-4ff0-9df2-21931ae71faa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:28.717485 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dfdb8c54-3e98-4ff0-9df2-21931ae71faa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:28.857918 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-dfdb8c54-3e98-4ff0-9df2-21931ae71faa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:28.866281 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-dfdb8c54-3e98-4ff0-9df2-21931ae71faa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1722 microversion: 2.1 time: 0.162557 Aug 21 01:29:28.866862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 281/551] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:28 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1722 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:29.886909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e9fd4b1d-68c0-4bb3-ac79-c00ab514b43e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:29.887590 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e9fd4b1d-68c0-4bb3-ac79-c00ab514b43e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:29.894647 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e9fd4b1d-68c0-4bb3-ac79-c00ab514b43e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:29.895047 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e9fd4b1d-68c0-4bb3-ac79-c00ab514b43e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:29.895666 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e9fd4b1d-68c0-4bb3-ac79-c00ab514b43e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:29.896401 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e9fd4b1d-68c0-4bb3-ac79-c00ab514b43e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:29.896901 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e9fd4b1d-68c0-4bb3-ac79-c00ab514b43e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:29.897365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e9fd4b1d-68c0-4bb3-ac79-c00ab514b43e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:30.035012 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e9fd4b1d-68c0-4bb3-ac79-c00ab514b43e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:30.049979 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e9fd4b1d-68c0-4bb3-ac79-c00ab514b43e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1722 microversion: 2.1 time: 0.166555 Aug 21 01:29:30.051958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 271/552] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:29 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1722 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:31.074363 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8d26e98f-4747-4c1d-8eed-27512b497ee3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:31.074983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8d26e98f-4747-4c1d-8eed-27512b497ee3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:31.082007 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d26e98f-4747-4c1d-8eed-27512b497ee3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:31.082813 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d26e98f-4747-4c1d-8eed-27512b497ee3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:31.083797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d26e98f-4747-4c1d-8eed-27512b497ee3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:31.083797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d26e98f-4747-4c1d-8eed-27512b497ee3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:31.084081 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d26e98f-4747-4c1d-8eed-27512b497ee3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:31.084534 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d26e98f-4747-4c1d-8eed-27512b497ee3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:31.230157 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8d26e98f-4747-4c1d-8eed-27512b497ee3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:31.238959 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8d26e98f-4747-4c1d-8eed-27512b497ee3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1722 microversion: 2.1 time: 0.170927 Aug 21 01:29:31.239543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 282/553] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:31 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1722 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:32.261283 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4a3025f9-a5ea-4521-a3a2-c3c5f75bd7f5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:32.261888 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4a3025f9-a5ea-4521-a3a2-c3c5f75bd7f5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:32.269866 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4a3025f9-a5ea-4521-a3a2-c3c5f75bd7f5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:32.270411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4a3025f9-a5ea-4521-a3a2-c3c5f75bd7f5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:32.270924 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4a3025f9-a5ea-4521-a3a2-c3c5f75bd7f5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:32.271703 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4a3025f9-a5ea-4521-a3a2-c3c5f75bd7f5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:32.272244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4a3025f9-a5ea-4521-a3a2-c3c5f75bd7f5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:32.272909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4a3025f9-a5ea-4521-a3a2-c3c5f75bd7f5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:32.409322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4a3025f9-a5ea-4521-a3a2-c3c5f75bd7f5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:32.417670 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4a3025f9-a5ea-4521-a3a2-c3c5f75bd7f5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1722 microversion: 2.1 time: 0.160256 Aug 21 01:29:32.418237 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 272/554] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:32 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1722 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:33.441201 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-41070b8c-42e0-4f0c-be98-627ff412aafa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:33.441932 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-41070b8c-42e0-4f0c-be98-627ff412aafa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:33.448452 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-41070b8c-42e0-4f0c-be98-627ff412aafa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:33.448988 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-41070b8c-42e0-4f0c-be98-627ff412aafa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:33.449424 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-41070b8c-42e0-4f0c-be98-627ff412aafa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:33.450120 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-41070b8c-42e0-4f0c-be98-627ff412aafa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:33.450564 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-41070b8c-42e0-4f0c-be98-627ff412aafa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:33.450983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-41070b8c-42e0-4f0c-be98-627ff412aafa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:33.592927 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-41070b8c-42e0-4f0c-be98-627ff412aafa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:33.599574 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-41070b8c-42e0-4f0c-be98-627ff412aafa tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1722 microversion: 2.1 time: 0.162798 Aug 21 01:29:33.600203 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 283/555] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:33 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1722 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:34.623153 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-068d31b3-dede-465e-a9ef-f6e8f17cca96 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:34.624233 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-068d31b3-dede-465e-a9ef-f6e8f17cca96 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:34.627955 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=56 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:34.635585 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-068d31b3-dede-465e-a9ef-f6e8f17cca96 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:34.636457 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-068d31b3-dede-465e-a9ef-f6e8f17cca96 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:34.637009 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-068d31b3-dede-465e-a9ef-f6e8f17cca96 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:34.638163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-068d31b3-dede-465e-a9ef-f6e8f17cca96 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:34.638971 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-068d31b3-dede-465e-a9ef-f6e8f17cca96 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:34.639479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-068d31b3-dede-465e-a9ef-f6e8f17cca96 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:34.789079 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-068d31b3-dede-465e-a9ef-f6e8f17cca96 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:34.799391 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-068d31b3-dede-465e-a9ef-f6e8f17cca96 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1728 microversion: 2.1 time: 0.182287 Aug 21 01:29:34.800055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 273/556] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:34 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1728 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:34.820966 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-31c9db36-ab81-4620-8be5-2c3fec6b06ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:34.821498 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-31c9db36-ab81-4620-8be5-2c3fec6b06ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:34.828542 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31c9db36-ab81-4620-8be5-2c3fec6b06ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:34.829039 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31c9db36-ab81-4620-8be5-2c3fec6b06ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:34.830878 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31c9db36-ab81-4620-8be5-2c3fec6b06ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:34.830878 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31c9db36-ab81-4620-8be5-2c3fec6b06ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:34.831222 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31c9db36-ab81-4620-8be5-2c3fec6b06ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:34.831676 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31c9db36-ab81-4620-8be5-2c3fec6b06ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:34.931955 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-31c9db36-ab81-4620-8be5-2c3fec6b06ba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.114511 Aug 21 01:29:34.933111 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 284/557] 10.222.0.47 () {58 vars in 1275 bytes} [Sun Aug 21 01:29:34 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-interface => generated 284 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:29:35.148396 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-57be02bd-bd54-43cd-a439-73094d75ed82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:29:35.151124 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-57be02bd-bd54-43cd-a439-73094d75ed82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:35.157924 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-57be02bd-bd54-43cd-a439-73094d75ed82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:35.158423 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-57be02bd-bd54-43cd-a439-73094d75ed82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:35.158861 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-57be02bd-bd54-43cd-a439-73094d75ed82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:35.159570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-57be02bd-bd54-43cd-a439-73094d75ed82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:35.160021 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-57be02bd-bd54-43cd-a439-73094d75ed82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:35.160473 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-57be02bd-bd54-43cd-a439-73094d75ed82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:35.465382 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-57be02bd-bd54-43cd-a439-73094d75ed82 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "POST /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/action" status: 202 len: 0 microversion: 2.1 time: 0.320376 Aug 21 01:29:35.469560 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 274/558] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:29:35 2022] POST /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/action => generated 0 bytes in 323 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:29:35.475840 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c906ec73-d797-42dc-9c94-e9709dc499ec tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:35.476851 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c906ec73-d797-42dc-9c94-e9709dc499ec tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:35.485218 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c906ec73-d797-42dc-9c94-e9709dc499ec tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:35.486000 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c906ec73-d797-42dc-9c94-e9709dc499ec tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:35.486994 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c906ec73-d797-42dc-9c94-e9709dc499ec tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:35.488602 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c906ec73-d797-42dc-9c94-e9709dc499ec tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:35.489750 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c906ec73-d797-42dc-9c94-e9709dc499ec tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:35.490779 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c906ec73-d797-42dc-9c94-e9709dc499ec tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:35.766641 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c906ec73-d797-42dc-9c94-e9709dc499ec tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:35.775755 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c906ec73-d797-42dc-9c94-e9709dc499ec tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1752 microversion: 2.1 time: 0.304696 Aug 21 01:29:35.776368 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 285/559] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:35 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1752 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:36.183637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-48727136-2fe5-4d74-b671-eef51e09b2a6 req-866ce6bf-9f9d-4a95-8137-9d833967d96f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eddb6b3f-8ca0-4f71-b371-19c6e65dd095", "tag": "1e55ecc3-f194-4529-b75c-be5293dba5c9"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:29:36.196826 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-48727136-2fe5-4d74-b671-eef51e09b2a6 req-866ce6bf-9f9d-4a95-8137-9d833967d96f service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:36.197432 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-48727136-2fe5-4d74-b671-eef51e09b2a6 req-866ce6bf-9f9d-4a95-8137-9d833967d96f service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:36.197935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-48727136-2fe5-4d74-b671-eef51e09b2a6 req-866ce6bf-9f9d-4a95-8137-9d833967d96f service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:36.213281 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-48727136-2fe5-4d74-b671-eef51e09b2a6 req-866ce6bf-9f9d-4a95-8137-9d833967d96f service nova] Creating event network-changed:1e55ecc3-f194-4529-b75c-be5293dba5c9 for instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 on hv-cinder-maste Aug 21 01:29:36.221435 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-48727136-2fe5-4d74-b671-eef51e09b2a6 req-866ce6bf-9f9d-4a95-8137-9d833967d96f service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042801 Aug 21 01:29:36.222045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 275/560] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:29:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:29:36.799561 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-292c3632-8757-492c-8625-1888d5e3723c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:36.806525 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-292c3632-8757-492c-8625-1888d5e3723c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:36.808964 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-292c3632-8757-492c-8625-1888d5e3723c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:36.809612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-292c3632-8757-492c-8625-1888d5e3723c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:36.810156 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-292c3632-8757-492c-8625-1888d5e3723c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:36.811270 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-292c3632-8757-492c-8625-1888d5e3723c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:36.811907 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-292c3632-8757-492c-8625-1888d5e3723c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:36.812601 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-292c3632-8757-492c-8625-1888d5e3723c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:36.996942 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-292c3632-8757-492c-8625-1888d5e3723c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:37.011115 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-292c3632-8757-492c-8625-1888d5e3723c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1752 microversion: 2.1 time: 0.216617 Aug 21 01:29:37.012053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 286/561] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:36 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1752 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:38.040861 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-5e6369aa-decf-41c2-935c-708f1774e92d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:38.041551 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-5e6369aa-decf-41c2-935c-708f1774e92d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:38.048502 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5e6369aa-decf-41c2-935c-708f1774e92d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:38.049010 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5e6369aa-decf-41c2-935c-708f1774e92d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:38.049462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5e6369aa-decf-41c2-935c-708f1774e92d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:38.050219 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5e6369aa-decf-41c2-935c-708f1774e92d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:38.050694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5e6369aa-decf-41c2-935c-708f1774e92d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:38.051123 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5e6369aa-decf-41c2-935c-708f1774e92d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:38.202075 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5e6369aa-decf-41c2-935c-708f1774e92d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:38.211188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5e6369aa-decf-41c2-935c-708f1774e92d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1752 microversion: 2.1 time: 0.174916 Aug 21 01:29:38.211890 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 276/562] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:38 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1752 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:38.235702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-ef41b88c-e642-4a6f-90a2-0a5ee41b8f24 req-70dde36e-c807-452a-b61c-5257594061a4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eddb6b3f-8ca0-4f71-b371-19c6e65dd095", "tag": "1e55ecc3-f194-4529-b75c-be5293dba5c9"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:29:38.246211 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ef41b88c-e642-4a6f-90a2-0a5ee41b8f24 req-70dde36e-c807-452a-b61c-5257594061a4 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:38.246712 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ef41b88c-e642-4a6f-90a2-0a5ee41b8f24 req-70dde36e-c807-452a-b61c-5257594061a4 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:38.247136 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ef41b88c-e642-4a6f-90a2-0a5ee41b8f24 req-70dde36e-c807-452a-b61c-5257594061a4 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:38.259639 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-ef41b88c-e642-4a6f-90a2-0a5ee41b8f24 req-70dde36e-c807-452a-b61c-5257594061a4 service nova] Creating event network-changed:1e55ecc3-f194-4529-b75c-be5293dba5c9 for instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 on hv-cinder-maste Aug 21 01:29:38.265290 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-ef41b88c-e642-4a6f-90a2-0a5ee41b8f24 req-70dde36e-c807-452a-b61c-5257594061a4 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032827 Aug 21 01:29:38.265969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 287/563] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:29:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:29:39.234063 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9bf0f0e6-512c-46c9-9a4a-dd92ea4e9117 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:39.235048 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9bf0f0e6-512c-46c9-9a4a-dd92ea4e9117 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:39.243564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9bf0f0e6-512c-46c9-9a4a-dd92ea4e9117 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:39.244046 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9bf0f0e6-512c-46c9-9a4a-dd92ea4e9117 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:39.244521 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9bf0f0e6-512c-46c9-9a4a-dd92ea4e9117 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:39.245212 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9bf0f0e6-512c-46c9-9a4a-dd92ea4e9117 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:39.245666 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9bf0f0e6-512c-46c9-9a4a-dd92ea4e9117 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:39.246116 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9bf0f0e6-512c-46c9-9a4a-dd92ea4e9117 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:39.464513 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9bf0f0e6-512c-46c9-9a4a-dd92ea4e9117 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:39.476818 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9bf0f0e6-512c-46c9-9a4a-dd92ea4e9117 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1752 microversion: 2.1 time: 0.247672 Aug 21 01:29:39.477460 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 277/564] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:39 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1752 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:40.499436 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3fa1f21e-ac93-4b7b-b022-06699c13fdf9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:40.500201 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3fa1f21e-ac93-4b7b-b022-06699c13fdf9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:40.508595 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3fa1f21e-ac93-4b7b-b022-06699c13fdf9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:40.509406 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3fa1f21e-ac93-4b7b-b022-06699c13fdf9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:40.510193 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3fa1f21e-ac93-4b7b-b022-06699c13fdf9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:40.511428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3fa1f21e-ac93-4b7b-b022-06699c13fdf9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:40.512416 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3fa1f21e-ac93-4b7b-b022-06699c13fdf9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:40.513202 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3fa1f21e-ac93-4b7b-b022-06699c13fdf9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:40.699584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3fa1f21e-ac93-4b7b-b022-06699c13fdf9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:40.708962 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3fa1f21e-ac93-4b7b-b022-06699c13fdf9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.213437 Aug 21 01:29:40.709769 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 288/565] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:40 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:41.733876 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-0b368378-1dbe-4dcb-a5c7-32e0f4e6d566 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:41.734734 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-0b368378-1dbe-4dcb-a5c7-32e0f4e6d566 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:41.742698 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0b368378-1dbe-4dcb-a5c7-32e0f4e6d566 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:41.743392 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0b368378-1dbe-4dcb-a5c7-32e0f4e6d566 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:41.743912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0b368378-1dbe-4dcb-a5c7-32e0f4e6d566 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:41.744700 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0b368378-1dbe-4dcb-a5c7-32e0f4e6d566 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:41.745212 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0b368378-1dbe-4dcb-a5c7-32e0f4e6d566 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:41.745732 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0b368378-1dbe-4dcb-a5c7-32e0f4e6d566 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:41.930717 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0b368378-1dbe-4dcb-a5c7-32e0f4e6d566 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:41.940019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-0b368378-1dbe-4dcb-a5c7-32e0f4e6d566 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.210032 Aug 21 01:29:41.940620 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 278/566] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:41 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:42.962266 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9f1fcb4b-b9cf-4479-8978-387f0c278680 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:42.962881 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9f1fcb4b-b9cf-4479-8978-387f0c278680 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:42.969382 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9f1fcb4b-b9cf-4479-8978-387f0c278680 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:42.969872 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9f1fcb4b-b9cf-4479-8978-387f0c278680 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:42.970326 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9f1fcb4b-b9cf-4479-8978-387f0c278680 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:42.971020 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9f1fcb4b-b9cf-4479-8978-387f0c278680 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:42.971477 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9f1fcb4b-b9cf-4479-8978-387f0c278680 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:42.971968 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9f1fcb4b-b9cf-4479-8978-387f0c278680 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:42.990316 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:43.123976 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9f1fcb4b-b9cf-4479-8978-387f0c278680 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:43.133163 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9f1fcb4b-b9cf-4479-8978-387f0c278680 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.174873 Aug 21 01:29:43.133805 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 289/567] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:42 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:43.227209 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-0c90c092-f356-46a0-b6c8-59c33f97cd81 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:43.228033 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-0c90c092-f356-46a0-b6c8-59c33f97cd81 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:43.234329 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0c90c092-f356-46a0-b6c8-59c33f97cd81 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:43.234815 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0c90c092-f356-46a0-b6c8-59c33f97cd81 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:43.235289 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0c90c092-f356-46a0-b6c8-59c33f97cd81 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:43.236550 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0c90c092-f356-46a0-b6c8-59c33f97cd81 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:43.237003 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0c90c092-f356-46a0-b6c8-59c33f97cd81 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:43.237432 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0c90c092-f356-46a0-b6c8-59c33f97cd81 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:43.332238 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-0c90c092-f356-46a0-b6c8-59c33f97cd81 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lazy-loading 'services' on Instance uuid 37524b56-3cc1-4276-802b-1419dee34d23 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:29:43.583350 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-0c90c092-f356-46a0-b6c8-59c33f97cd81 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "DELETE /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23/os-volume_attachments/b4192168-4093-4570-b7e8-d4767e661456" status: 202 len: 0 microversion: 2.1 time: 0.360698 Aug 21 01:29:43.583941 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 279/568] 10.222.0.47 () {58 vars in 1416 bytes} [Sun Aug 21 01:29:43 2022] DELETE /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23/os-volume_attachments/b4192168-4093-4570-b7e8-d4767e661456 => generated 0 bytes in 362 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:29:44.155140 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c9835daf-90ae-466c-946d-304f509bff9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:44.155705 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c9835daf-90ae-466c-946d-304f509bff9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:44.163078 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c9835daf-90ae-466c-946d-304f509bff9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:44.163568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c9835daf-90ae-466c-946d-304f509bff9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:44.164004 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c9835daf-90ae-466c-946d-304f509bff9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:44.164747 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c9835daf-90ae-466c-946d-304f509bff9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:44.165206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c9835daf-90ae-466c-946d-304f509bff9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:44.165884 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c9835daf-90ae-466c-946d-304f509bff9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:44.327775 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c9835daf-90ae-466c-946d-304f509bff9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:44.337176 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c9835daf-90ae-466c-946d-304f509bff9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.185935 Aug 21 01:29:44.337782 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 290/569] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:44 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:45.361721 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-0a37a8a2-9d7c-4437-9d7d-2bc0eb43e89e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:45.362517 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-0a37a8a2-9d7c-4437-9d7d-2bc0eb43e89e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:45.369393 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0a37a8a2-9d7c-4437-9d7d-2bc0eb43e89e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:45.370062 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0a37a8a2-9d7c-4437-9d7d-2bc0eb43e89e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:45.370597 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0a37a8a2-9d7c-4437-9d7d-2bc0eb43e89e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:45.371432 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0a37a8a2-9d7c-4437-9d7d-2bc0eb43e89e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:45.371990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0a37a8a2-9d7c-4437-9d7d-2bc0eb43e89e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:45.372538 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0a37a8a2-9d7c-4437-9d7d-2bc0eb43e89e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:45.537073 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0a37a8a2-9d7c-4437-9d7d-2bc0eb43e89e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:45.547378 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-0a37a8a2-9d7c-4437-9d7d-2bc0eb43e89e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.190398 Aug 21 01:29:45.547961 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 280/570] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:45 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:45.814018 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-26eba43c-e109-4118-b876-1df6db44fbca tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:45.815414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-26eba43c-e109-4118-b876-1df6db44fbca tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:45.824140 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-26eba43c-e109-4118-b876-1df6db44fbca tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:45.825056 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-26eba43c-e109-4118-b876-1df6db44fbca tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:45.825448 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-26eba43c-e109-4118-b876-1df6db44fbca tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:45.826350 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-26eba43c-e109-4118-b876-1df6db44fbca tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:45.827130 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-26eba43c-e109-4118-b876-1df6db44fbca tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:45.827695 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-26eba43c-e109-4118-b876-1df6db44fbca tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:45.858907 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-26eba43c-e109-4118-b876-1df6db44fbca tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Going to try to terminate instance {{(pid=121277) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:29:46.190504 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-26eba43c-e109-4118-b876-1df6db44fbca tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "DELETE /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 204 len: 0 microversion: 2.1 time: 0.380254 Aug 21 01:29:46.191030 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 291/571] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:29:45 2022] DELETE /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 21 01:29:46.199848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-81947e9b-af66-4364-b13a-1e524dd48eef tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:46.200818 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-81947e9b-af66-4364-b13a-1e524dd48eef tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:46.206595 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81947e9b-af66-4364-b13a-1e524dd48eef tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:46.207070 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81947e9b-af66-4364-b13a-1e524dd48eef tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:46.207509 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81947e9b-af66-4364-b13a-1e524dd48eef tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:46.208232 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81947e9b-af66-4364-b13a-1e524dd48eef tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:46.208686 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81947e9b-af66-4364-b13a-1e524dd48eef tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:46.209170 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81947e9b-af66-4364-b13a-1e524dd48eef tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:46.372067 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-81947e9b-af66-4364-b13a-1e524dd48eef tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:46.380656 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-81947e9b-af66-4364-b13a-1e524dd48eef tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1515 microversion: 2.1 time: 0.184950 Aug 21 01:29:46.381218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 281/572] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:46 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1515 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:46.570476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c62c94b3-6902-4c15-bac1-5e29a7f1c54f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:46.571208 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c62c94b3-6902-4c15-bac1-5e29a7f1c54f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:46.577528 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c62c94b3-6902-4c15-bac1-5e29a7f1c54f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:46.578046 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c62c94b3-6902-4c15-bac1-5e29a7f1c54f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:46.578487 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c62c94b3-6902-4c15-bac1-5e29a7f1c54f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:46.579183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c62c94b3-6902-4c15-bac1-5e29a7f1c54f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:46.579640 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c62c94b3-6902-4c15-bac1-5e29a7f1c54f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:46.580095 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c62c94b3-6902-4c15-bac1-5e29a7f1c54f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:46.738182 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c62c94b3-6902-4c15-bac1-5e29a7f1c54f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:46.747256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c62c94b3-6902-4c15-bac1-5e29a7f1c54f tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.181374 Aug 21 01:29:46.747836 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 292/573] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:46 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:47.402425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-28838341-358a-4d8d-98d7-edb163d41367 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:47.403216 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-28838341-358a-4d8d-98d7-edb163d41367 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:47.413271 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28838341-358a-4d8d-98d7-edb163d41367 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:47.413760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28838341-358a-4d8d-98d7-edb163d41367 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:47.414194 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28838341-358a-4d8d-98d7-edb163d41367 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:47.414889 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28838341-358a-4d8d-98d7-edb163d41367 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:47.415337 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28838341-358a-4d8d-98d7-edb163d41367 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:47.415757 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-28838341-358a-4d8d-98d7-edb163d41367 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:47.559052 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-28838341-358a-4d8d-98d7-edb163d41367 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:47.567959 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-28838341-358a-4d8d-98d7-edb163d41367 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1515 microversion: 2.1 time: 0.169601 Aug 21 01:29:47.568841 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 282/574] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:47 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1515 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:47.768889 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-fdf11474-565e-4633-ab17-5651e3691d3d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:47.769740 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-fdf11474-565e-4633-ab17-5651e3691d3d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:47.776699 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fdf11474-565e-4633-ab17-5651e3691d3d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:47.777562 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fdf11474-565e-4633-ab17-5651e3691d3d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:47.778331 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fdf11474-565e-4633-ab17-5651e3691d3d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:47.779548 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fdf11474-565e-4633-ab17-5651e3691d3d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:47.780441 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fdf11474-565e-4633-ab17-5651e3691d3d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:47.781210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fdf11474-565e-4633-ab17-5651e3691d3d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:47.947256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-fdf11474-565e-4633-ab17-5651e3691d3d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:47.959706 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-fdf11474-565e-4633-ab17-5651e3691d3d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.194756 Aug 21 01:29:47.960358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 293/575] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:47 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:48.593149 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-55fdbfef-382c-42a5-9d71-a7c586c6cada tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:48.593915 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-55fdbfef-382c-42a5-9d71-a7c586c6cada tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:48.601978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55fdbfef-382c-42a5-9d71-a7c586c6cada tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:48.602618 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55fdbfef-382c-42a5-9d71-a7c586c6cada tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:48.603175 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55fdbfef-382c-42a5-9d71-a7c586c6cada tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:48.604436 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55fdbfef-382c-42a5-9d71-a7c586c6cada tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:48.605300 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55fdbfef-382c-42a5-9d71-a7c586c6cada tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:48.605998 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55fdbfef-382c-42a5-9d71-a7c586c6cada tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:48.874580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-55fdbfef-382c-42a5-9d71-a7c586c6cada tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:48.888946 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-55fdbfef-382c-42a5-9d71-a7c586c6cada tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1515 microversion: 2.1 time: 0.300006 Aug 21 01:29:48.903248 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 283/576] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:48 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1515 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:48.985957 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-fbe7be18-44e2-44a9-b363-5afc3ea49cdc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:48.986822 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-fbe7be18-44e2-44a9-b363-5afc3ea49cdc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:48.989337 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:48.994736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fbe7be18-44e2-44a9-b363-5afc3ea49cdc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:48.995233 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fbe7be18-44e2-44a9-b363-5afc3ea49cdc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:48.995705 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fbe7be18-44e2-44a9-b363-5afc3ea49cdc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:48.996476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fbe7be18-44e2-44a9-b363-5afc3ea49cdc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:48.996934 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fbe7be18-44e2-44a9-b363-5afc3ea49cdc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:48.997366 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fbe7be18-44e2-44a9-b363-5afc3ea49cdc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:49.326320 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-fbe7be18-44e2-44a9-b363-5afc3ea49cdc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:49.335219 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-fbe7be18-44e2-44a9-b363-5afc3ea49cdc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.354724 Aug 21 01:29:49.335819 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 294/577] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:48 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:49.432820 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-f7542c55-7778-4c58-9dc8-66a0b525510c req-ac5a5adc-c0af-463d-ae7a-8752e87201ec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "37524b56-3cc1-4276-802b-1419dee34d23", "name": "network-vif-deleted", "tag": "23ea1b5c-1202-43ad-af9f-6cc34179339a"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:29:49.444016 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-f7542c55-7778-4c58-9dc8-66a0b525510c req-ac5a5adc-c0af-463d-ae7a-8752e87201ec service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:49.444605 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-f7542c55-7778-4c58-9dc8-66a0b525510c req-ac5a5adc-c0af-463d-ae7a-8752e87201ec service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:49.445030 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-f7542c55-7778-4c58-9dc8-66a0b525510c req-ac5a5adc-c0af-463d-ae7a-8752e87201ec service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:49.456995 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-f7542c55-7778-4c58-9dc8-66a0b525510c req-ac5a5adc-c0af-463d-ae7a-8752e87201ec service nova] Creating event network-vif-deleted:23ea1b5c-1202-43ad-af9f-6cc34179339a for instance 37524b56-3cc1-4276-802b-1419dee34d23 on hv-cinder-maste Aug 21 01:29:49.462144 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-f7542c55-7778-4c58-9dc8-66a0b525510c req-ac5a5adc-c0af-463d-ae7a-8752e87201ec service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033839 Aug 21 01:29:49.462684 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 284/578] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:29:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:29:49.913509 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a892e467-2626-481a-98e1-8cc94031e325 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:49.914116 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a892e467-2626-481a-98e1-8cc94031e325 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:49.922916 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a892e467-2626-481a-98e1-8cc94031e325 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:49.923398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a892e467-2626-481a-98e1-8cc94031e325 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:49.923831 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a892e467-2626-481a-98e1-8cc94031e325 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:49.936518 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a892e467-2626-481a-98e1-8cc94031e325 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:49.936518 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a892e467-2626-481a-98e1-8cc94031e325 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:49.936518 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a892e467-2626-481a-98e1-8cc94031e325 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:49.997389 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a892e467-2626-481a-98e1-8cc94031e325 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:50.006372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a892e467-2626-481a-98e1-8cc94031e325 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 200 len: 1304 microversion: 2.1 time: 0.096603 Aug 21 01:29:50.006946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 295/579] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:49 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 1304 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:50.362293 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8482bf76-513a-41a3-87f5-dfa49c369f0e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:50.362987 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8482bf76-513a-41a3-87f5-dfa49c369f0e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:50.370128 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8482bf76-513a-41a3-87f5-dfa49c369f0e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:50.370610 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8482bf76-513a-41a3-87f5-dfa49c369f0e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:50.371044 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8482bf76-513a-41a3-87f5-dfa49c369f0e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:50.371731 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8482bf76-513a-41a3-87f5-dfa49c369f0e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:50.372206 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8482bf76-513a-41a3-87f5-dfa49c369f0e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:50.372627 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8482bf76-513a-41a3-87f5-dfa49c369f0e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:50.391700 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:50.548215 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8482bf76-513a-41a3-87f5-dfa49c369f0e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:50.557646 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8482bf76-513a-41a3-87f5-dfa49c369f0e tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.200013 Aug 21 01:29:50.560138 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 285/580] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:50 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:51.027236 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b160bb5d-9ffc-44e1-940e-159896741ce5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:51.027797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b160bb5d-9ffc-44e1-940e-159896741ce5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: 37524b56-3cc1-4276-802b-1419dee34d23] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:51.034940 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b160bb5d-9ffc-44e1-940e-159896741ce5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:51.035445 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b160bb5d-9ffc-44e1-940e-159896741ce5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:51.035892 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b160bb5d-9ffc-44e1-940e-159896741ce5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:51.036644 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b160bb5d-9ffc-44e1-940e-159896741ce5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:51.037106 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b160bb5d-9ffc-44e1-940e-159896741ce5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:51.037539 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b160bb5d-9ffc-44e1-940e-159896741ce5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:51.052024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-b160bb5d-9ffc-44e1-940e-159896741ce5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] HTTP exception thrown: Instance 37524b56-3cc1-4276-802b-1419dee34d23 could not be found. Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b160bb5d-9ffc-44e1-940e-159896741ce5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Request method failure captured: Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 HTTP/1.1 Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYpCUMdYFiBnjXzaa6PTyGyJsWfhcnlyjAzkjhPHAIWMJQEHncMLbhc8nFjmquAGwoV9oJnuJyERqn73N5Fn6fuoLFlkJv7tOQzLxl6tpfweEYS_LkReWnpfFGhQyNsYJu6J_MYOEXBiOWmTaYlit6pWtpxTXEYRlvUkjWkE6uNLBhg Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 67f492f96d314cfea70bdbfa3caaee9e Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1041524239 Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:29:51.053726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:29:51.055198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/67f492f96d314cfea70bdbfa3caaee9e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_67f492f96d314cfea70bdbfa3caaee9e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/67f492f96d314cfea70bdbfa3caaee9e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/67f492f96d314cfea70bdbfa3caaee9e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:29:51.055198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-VolumesSnapshotTestJSON-1041524239 Aug 21 01:29:51.055198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 67f492f96d314cfea70bdbfa3caaee9e Aug 21 01:29:51.055198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1041524239 Aug 21 01:29:51.055198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-VolumesSnapshotTestJSON-1041524239-project Aug 21 01:29:51.055198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:29:51.055198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:29:51.055198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 2325a3d547d043d2a9ac95986187027b Aug 21 01:29:51.055198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-VolumesSnapshotTestJSON-1041524239-project Aug 21 01:29:51.055198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:29:51.055198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 37524b56-3cc1-4276-802b-1419dee34d23 could not be found. Aug 21 01:29:51.055198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '37524b56-3cc1-4276-802b-1419dee34d23'} Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 37524b56-3cc1-4276-802b-1419dee34d23 could not be found. Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:29:51.056194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 37524b56-3cc1-4276-802b-1419dee34d23 could not be found. Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 37524b56-3cc1-4276-802b-1419dee34d23 could not be found. Aug 21 01:29:51.057704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:29:51.058969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b160bb5d-9ffc-44e1-940e-159896741ce5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Returning 404 to user: Instance 37524b56-3cc1-4276-802b-1419dee34d23 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:29:51.058969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b160bb5d-9ffc-44e1-940e-159896741ce5 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23" status: 404 len: 111 microversion: 2.1 time: 0.032074 Aug 21 01:29:51.059230 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 296/581] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:51 2022] GET /compute/v2.1/servers/37524b56-3cc1-4276-802b-1419dee34d23 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:29:51.578799 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-82d3901f-7ff7-4bf8-8b5d-3b199b5efb9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:51.579452 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-82d3901f-7ff7-4bf8-8b5d-3b199b5efb9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:51.586397 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-82d3901f-7ff7-4bf8-8b5d-3b199b5efb9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:51.586892 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-82d3901f-7ff7-4bf8-8b5d-3b199b5efb9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:51.587323 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-82d3901f-7ff7-4bf8-8b5d-3b199b5efb9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:51.588032 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-82d3901f-7ff7-4bf8-8b5d-3b199b5efb9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:51.593430 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-82d3901f-7ff7-4bf8-8b5d-3b199b5efb9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:51.593430 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-82d3901f-7ff7-4bf8-8b5d-3b199b5efb9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:51.738391 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-82d3901f-7ff7-4bf8-8b5d-3b199b5efb9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:51.747047 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-82d3901f-7ff7-4bf8-8b5d-3b199b5efb9c tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.172004 Aug 21 01:29:51.747632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 286/582] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:51 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:52.768552 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-557cb420-aa1d-4c5e-b12d-f32d37ff1c01 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:52.769163 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-557cb420-aa1d-4c5e-b12d-f32d37ff1c01 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:52.775483 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-557cb420-aa1d-4c5e-b12d-f32d37ff1c01 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:52.775959 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-557cb420-aa1d-4c5e-b12d-f32d37ff1c01 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:52.776417 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-557cb420-aa1d-4c5e-b12d-f32d37ff1c01 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:52.777101 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-557cb420-aa1d-4c5e-b12d-f32d37ff1c01 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:52.777550 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-557cb420-aa1d-4c5e-b12d-f32d37ff1c01 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:52.777966 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-557cb420-aa1d-4c5e-b12d-f32d37ff1c01 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:52.922535 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-557cb420-aa1d-4c5e-b12d-f32d37ff1c01 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:52.931626 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-557cb420-aa1d-4c5e-b12d-f32d37ff1c01 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.167542 Aug 21 01:29:52.932287 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 297/583] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:52 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:53.953852 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-554a8c0d-cf63-4276-9d81-a75d67086c25 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:53.954466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-554a8c0d-cf63-4276-9d81-a75d67086c25 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:53.961705 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-554a8c0d-cf63-4276-9d81-a75d67086c25 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:53.962184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-554a8c0d-cf63-4276-9d81-a75d67086c25 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:53.962617 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-554a8c0d-cf63-4276-9d81-a75d67086c25 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:53.963300 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-554a8c0d-cf63-4276-9d81-a75d67086c25 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:53.963749 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-554a8c0d-cf63-4276-9d81-a75d67086c25 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:53.964192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-554a8c0d-cf63-4276-9d81-a75d67086c25 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:54.110397 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-554a8c0d-cf63-4276-9d81-a75d67086c25 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:54.118515 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-554a8c0d-cf63-4276-9d81-a75d67086c25 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.168705 Aug 21 01:29:54.119074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 287/584] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:53 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:55.006645 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3bb2ac7f-31f5-4f92-a955-0769f0b65c1a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:55.007210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3bb2ac7f-31f5-4f92-a955-0769f0b65c1a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:55.013921 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3bb2ac7f-31f5-4f92-a955-0769f0b65c1a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:55.014408 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3bb2ac7f-31f5-4f92-a955-0769f0b65c1a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:55.014847 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3bb2ac7f-31f5-4f92-a955-0769f0b65c1a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:55.015545 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3bb2ac7f-31f5-4f92-a955-0769f0b65c1a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:55.016042 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3bb2ac7f-31f5-4f92-a955-0769f0b65c1a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:55.016565 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3bb2ac7f-31f5-4f92-a955-0769f0b65c1a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:55.049360 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3bb2ac7f-31f5-4f92-a955-0769f0b65c1a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-volume_attachments/49b53473-b9ab-4815-8822-ed1330e3657c" status: 200 len: 194 microversion: 2.20 time: 0.046844 Aug 21 01:29:55.049966 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 298/585] 10.222.0.47 () {60 vars in 1454 bytes} [Sun Aug 21 01:29:55 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-volume_attachments/49b53473-b9ab-4815-8822-ed1330e3657c => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:29:55.118221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-47e842ed-9cfd-45b5-8988-16589a084a2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:55.118728 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-47e842ed-9cfd-45b5-8988-16589a084a2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:55.125258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-47e842ed-9cfd-45b5-8988-16589a084a2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:55.125739 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-47e842ed-9cfd-45b5-8988-16589a084a2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:55.126175 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-47e842ed-9cfd-45b5-8988-16589a084a2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:55.126889 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-47e842ed-9cfd-45b5-8988-16589a084a2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:55.127422 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-47e842ed-9cfd-45b5-8988-16589a084a2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:55.127845 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-47e842ed-9cfd-45b5-8988-16589a084a2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:55.139782 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-05e0b105-ca79-45f9-a0c0-1fba403833d1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:55.140494 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-05e0b105-ca79-45f9-a0c0-1fba403833d1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:55.146588 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-05e0b105-ca79-45f9-a0c0-1fba403833d1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:55.147073 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-05e0b105-ca79-45f9-a0c0-1fba403833d1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:55.147507 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-05e0b105-ca79-45f9-a0c0-1fba403833d1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:55.148216 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-05e0b105-ca79-45f9-a0c0-1fba403833d1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:55.148667 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-05e0b105-ca79-45f9-a0c0-1fba403833d1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:55.149206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-05e0b105-ca79-45f9-a0c0-1fba403833d1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:55.212064 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-47e842ed-9cfd-45b5-8988-16589a084a2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lazy-loading 'services' on Instance uuid 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:29:55.295112 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-05e0b105-ca79-45f9-a0c0-1fba403833d1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:55.302872 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-05e0b105-ca79-45f9-a0c0-1fba403833d1 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.166278 Aug 21 01:29:55.303438 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 299/586] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:55 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:55.464726 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-47e842ed-9cfd-45b5-8988-16589a084a2e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "DELETE /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-volume_attachments/49b53473-b9ab-4815-8822-ed1330e3657c" status: 202 len: 0 microversion: 2.20 time: 0.350962 Aug 21 01:29:55.465317 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 288/587] 10.222.0.47 () {60 vars in 1457 bytes} [Sun Aug 21 01:29:55 2022] DELETE /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-volume_attachments/49b53473-b9ab-4815-8822-ed1330e3657c => generated 0 bytes in 352 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:29:55.474001 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-1a4d0254-56a5-4711-8eae-090651c0efb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:55.477011 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-1a4d0254-56a5-4711-8eae-090651c0efb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:55.483206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1a4d0254-56a5-4711-8eae-090651c0efb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:55.483715 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1a4d0254-56a5-4711-8eae-090651c0efb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:55.484186 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1a4d0254-56a5-4711-8eae-090651c0efb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:55.484952 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1a4d0254-56a5-4711-8eae-090651c0efb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:55.485435 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1a4d0254-56a5-4711-8eae-090651c0efb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:55.485866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1a4d0254-56a5-4711-8eae-090651c0efb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:55.515974 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-1a4d0254-56a5-4711-8eae-090651c0efb3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046068 Aug 21 01:29:55.516476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 300/588] 10.222.0.47 () {60 vars in 1343 bytes} [Sun Aug 21 01:29:55 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:29:56.325776 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7ce7d669-f3fe-492a-af69-da8704eb6866 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:56.326388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7ce7d669-f3fe-492a-af69-da8704eb6866 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:56.333311 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7ce7d669-f3fe-492a-af69-da8704eb6866 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:56.333793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7ce7d669-f3fe-492a-af69-da8704eb6866 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:56.334240 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7ce7d669-f3fe-492a-af69-da8704eb6866 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:56.334939 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7ce7d669-f3fe-492a-af69-da8704eb6866 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:56.335386 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7ce7d669-f3fe-492a-af69-da8704eb6866 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:56.335806 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7ce7d669-f3fe-492a-af69-da8704eb6866 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:56.484776 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7ce7d669-f3fe-492a-af69-da8704eb6866 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:56.494828 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7ce7d669-f3fe-492a-af69-da8704eb6866 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.173851 Aug 21 01:29:56.496418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 289/589] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:56 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:56.528483 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-63c13bd8-ccd2-4735-a2b2-9432405d11eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:56.530677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-63c13bd8-ccd2-4735-a2b2-9432405d11eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:56.537947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63c13bd8-ccd2-4735-a2b2-9432405d11eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:56.542802 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63c13bd8-ccd2-4735-a2b2-9432405d11eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:56.542802 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63c13bd8-ccd2-4735-a2b2-9432405d11eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:56.542802 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63c13bd8-ccd2-4735-a2b2-9432405d11eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:56.542802 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63c13bd8-ccd2-4735-a2b2-9432405d11eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:56.543193 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-63c13bd8-ccd2-4735-a2b2-9432405d11eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:56.572422 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-63c13bd8-ccd2-4735-a2b2-9432405d11eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047986 Aug 21 01:29:56.572979 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 301/590] 10.222.0.47 () {60 vars in 1343 bytes} [Sun Aug 21 01:29:56 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:29:57.521832 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ae7881e3-de1c-4ba3-ab7d-bf268d3f6138 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:57.521832 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ae7881e3-de1c-4ba3-ab7d-bf268d3f6138 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:57.524733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae7881e3-de1c-4ba3-ab7d-bf268d3f6138 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:57.525139 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae7881e3-de1c-4ba3-ab7d-bf268d3f6138 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:57.525572 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae7881e3-de1c-4ba3-ab7d-bf268d3f6138 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:57.526339 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae7881e3-de1c-4ba3-ab7d-bf268d3f6138 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:57.526712 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae7881e3-de1c-4ba3-ab7d-bf268d3f6138 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:57.527130 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae7881e3-de1c-4ba3-ab7d-bf268d3f6138 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:57.585986 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-76dedebf-4f25-4a5b-8ab4-1aadff1bea31 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:57.588238 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-76dedebf-4f25-4a5b-8ab4-1aadff1bea31 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:57.597106 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-76dedebf-4f25-4a5b-8ab4-1aadff1bea31 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:57.597601 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-76dedebf-4f25-4a5b-8ab4-1aadff1bea31 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:57.598031 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-76dedebf-4f25-4a5b-8ab4-1aadff1bea31 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:57.598741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-76dedebf-4f25-4a5b-8ab4-1aadff1bea31 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:57.599210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-76dedebf-4f25-4a5b-8ab4-1aadff1bea31 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:57.599633 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-76dedebf-4f25-4a5b-8ab4-1aadff1bea31 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:57.635903 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-76dedebf-4f25-4a5b-8ab4-1aadff1bea31 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.054748 Aug 21 01:29:57.639229 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 302/591] 10.222.0.47 () {60 vars in 1343 bytes} [Sun Aug 21 01:29:57 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1/os-volume_attachments => generated 25 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:29:57.680870 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ae7881e3-de1c-4ba3-ab7d-bf268d3f6138 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:57.690022 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ae7881e3-de1c-4ba3-ab7d-bf268d3f6138 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.177847 Aug 21 01:29:57.696864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 290/592] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:57 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:57.704180 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-545584e4-1e25-4698-af6d-3877a9a9d529 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:57.704672 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-545584e4-1e25-4698-af6d-3877a9a9d529 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:57.710709 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-545584e4-1e25-4698-af6d-3877a9a9d529 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:57.711187 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-545584e4-1e25-4698-af6d-3877a9a9d529 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:57.711639 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-545584e4-1e25-4698-af6d-3877a9a9d529 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:57.712337 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-545584e4-1e25-4698-af6d-3877a9a9d529 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:57.712794 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-545584e4-1e25-4698-af6d-3877a9a9d529 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:57.713231 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-545584e4-1e25-4698-af6d-3877a9a9d529 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:57.737929 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-545584e4-1e25-4698-af6d-3877a9a9d529 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Going to try to terminate instance {{(pid=121277) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:29:57.909425 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:29:58.262538 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-545584e4-1e25-4698-af6d-3877a9a9d529 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "DELETE /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 204 len: 0 microversion: 2.20 time: 0.561924 Aug 21 01:29:58.263144 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 303/593] 10.222.0.47 () {60 vars in 1280 bytes} [Sun Aug 21 01:29:57 2022] DELETE /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 21 01:29:58.271348 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1c1dffcf-e305-4f2b-be04-bc6a02d010e8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:58.271959 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1c1dffcf-e305-4f2b-be04-bc6a02d010e8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:58.278158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c1dffcf-e305-4f2b-be04-bc6a02d010e8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:58.278639 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c1dffcf-e305-4f2b-be04-bc6a02d010e8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:58.279080 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c1dffcf-e305-4f2b-be04-bc6a02d010e8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:58.279769 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c1dffcf-e305-4f2b-be04-bc6a02d010e8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:58.280252 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c1dffcf-e305-4f2b-be04-bc6a02d010e8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:58.280679 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c1dffcf-e305-4f2b-be04-bc6a02d010e8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:58.415522 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1c1dffcf-e305-4f2b-be04-bc6a02d010e8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:58.424924 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1c1dffcf-e305-4f2b-be04-bc6a02d010e8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:58.426430 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1c1dffcf-e305-4f2b-be04-bc6a02d010e8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:58.427606 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1c1dffcf-e305-4f2b-be04-bc6a02d010e8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.160263 Aug 21 01:29:58.428193 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 291/594] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:29:58 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:29:58.713203 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-44157406-9743-4bd2-bdbc-1fca17bbac95 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:58.713899 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-44157406-9743-4bd2-bdbc-1fca17bbac95 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:58.720336 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-44157406-9743-4bd2-bdbc-1fca17bbac95 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:58.720816 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-44157406-9743-4bd2-bdbc-1fca17bbac95 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:58.721250 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-44157406-9743-4bd2-bdbc-1fca17bbac95 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:58.721931 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-44157406-9743-4bd2-bdbc-1fca17bbac95 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:58.722378 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-44157406-9743-4bd2-bdbc-1fca17bbac95 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:58.722882 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-44157406-9743-4bd2-bdbc-1fca17bbac95 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:58.865982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-44157406-9743-4bd2-bdbc-1fca17bbac95 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:58.874534 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-44157406-9743-4bd2-bdbc-1fca17bbac95 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.166121 Aug 21 01:29:58.875110 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 304/595] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:58 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:29:59.451930 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ed82b23a-f87c-4b1b-8ce0-35b49e37c93a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:59.452769 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ed82b23a-f87c-4b1b-8ce0-35b49e37c93a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:59.459310 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ed82b23a-f87c-4b1b-8ce0-35b49e37c93a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:59.459796 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ed82b23a-f87c-4b1b-8ce0-35b49e37c93a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:59.460520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ed82b23a-f87c-4b1b-8ce0-35b49e37c93a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:59.461251 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ed82b23a-f87c-4b1b-8ce0-35b49e37c93a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:59.461705 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ed82b23a-f87c-4b1b-8ce0-35b49e37c93a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:59.462129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ed82b23a-f87c-4b1b-8ce0-35b49e37c93a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:59.593634 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ed82b23a-f87c-4b1b-8ce0-35b49e37c93a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:59.602939 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ed82b23a-f87c-4b1b-8ce0-35b49e37c93a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:59.604492 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ed82b23a-f87c-4b1b-8ce0-35b49e37c93a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:29:59.605655 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ed82b23a-f87c-4b1b-8ce0-35b49e37c93a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.157600 Aug 21 01:29:59.606201 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 292/596] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:29:59 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:29:59.896510 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d741e2b7-42cf-4e16-a46d-bfda3f10387d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:29:59.897076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d741e2b7-42cf-4e16-a46d-bfda3f10387d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:29:59.903939 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d741e2b7-42cf-4e16-a46d-bfda3f10387d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:59.904452 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d741e2b7-42cf-4e16-a46d-bfda3f10387d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:59.904887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d741e2b7-42cf-4e16-a46d-bfda3f10387d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:29:59.905571 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d741e2b7-42cf-4e16-a46d-bfda3f10387d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:29:59.906020 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d741e2b7-42cf-4e16-a46d-bfda3f10387d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:29:59.906440 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d741e2b7-42cf-4e16-a46d-bfda3f10387d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:00.103714 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d741e2b7-42cf-4e16-a46d-bfda3f10387d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:00.112387 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d741e2b7-42cf-4e16-a46d-bfda3f10387d tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.219755 Aug 21 01:30:00.112958 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 305/597] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:29:59 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:00.630702 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9c277af8-01dc-4b66-9ad5-c85b6b5c035d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:00.631189 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9c277af8-01dc-4b66-9ad5-c85b6b5c035d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:00.638631 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c277af8-01dc-4b66-9ad5-c85b6b5c035d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:00.639124 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c277af8-01dc-4b66-9ad5-c85b6b5c035d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:00.639562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c277af8-01dc-4b66-9ad5-c85b6b5c035d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:00.640300 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c277af8-01dc-4b66-9ad5-c85b6b5c035d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:00.640756 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c277af8-01dc-4b66-9ad5-c85b6b5c035d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:00.641188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c277af8-01dc-4b66-9ad5-c85b6b5c035d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:00.823288 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9c277af8-01dc-4b66-9ad5-c85b6b5c035d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:00.831661 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9c277af8-01dc-4b66-9ad5-c85b6b5c035d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:00.833283 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9c277af8-01dc-4b66-9ad5-c85b6b5c035d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:00.834624 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9c277af8-01dc-4b66-9ad5-c85b6b5c035d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.208052 Aug 21 01:30:00.834983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 293/598] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:00 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:30:01.134601 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c8146439-38c8-46f3-899b-7ca2f56abe52 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:01.135210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c8146439-38c8-46f3-899b-7ca2f56abe52 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:01.141850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c8146439-38c8-46f3-899b-7ca2f56abe52 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:01.142332 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c8146439-38c8-46f3-899b-7ca2f56abe52 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:01.142769 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c8146439-38c8-46f3-899b-7ca2f56abe52 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:01.143459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c8146439-38c8-46f3-899b-7ca2f56abe52 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:01.143913 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c8146439-38c8-46f3-899b-7ca2f56abe52 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:01.144399 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c8146439-38c8-46f3-899b-7ca2f56abe52 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:01.290759 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c8146439-38c8-46f3-899b-7ca2f56abe52 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:01.299172 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c8146439-38c8-46f3-899b-7ca2f56abe52 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.168928 Aug 21 01:30:01.302210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 306/599] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:01 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:01.710921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1624153887", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "networks": [{"uuid": "5a72733d-0e4b-47b7-9270-d8de517ee151"}]}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:01.763928 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] validate_networks() for [('5a72733d-0e4b-47b7-9270-d8de517ee151', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:30:01.857435 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-0b4aad86-4688-4234-bf44-281e2d24e25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:01.858063 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-0b4aad86-4688-4234-bf44-281e2d24e25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:01.864078 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0b4aad86-4688-4234-bf44-281e2d24e25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:01.864589 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0b4aad86-4688-4234-bf44-281e2d24e25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:01.865023 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0b4aad86-4688-4234-bf44-281e2d24e25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:01.865706 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0b4aad86-4688-4234-bf44-281e2d24e25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:01.866154 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0b4aad86-4688-4234-bf44-281e2d24e25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:01.866572 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0b4aad86-4688-4234-bf44-281e2d24e25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:02.093459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-0b4aad86-4688-4234-bf44-281e2d24e25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:02.101912 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-0b4aad86-4688-4234-bf44-281e2d24e25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:02.103417 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-0b4aad86-4688-4234-bf44-281e2d24e25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:02.104599 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-0b4aad86-4688-4234-bf44-281e2d24e25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.251162 Aug 21 01:30:02.105158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 307/600] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:01 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:30:02.181311 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:30:02.181936 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:30:02.182372 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:30:02.182773 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:30:02.183214 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:30:02.191863 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:02.192373 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:02.192807 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:02.193768 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:02.194275 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:02.194632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:02.211473 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Getting quotas for project 67f492f96d314cfea70bdbfa3caaee9e. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:30:02.215379 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Getting quotas for user 2325a3d547d043d2a9ac95986187027b and project 67f492f96d314cfea70bdbfa3caaee9e. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:30:02.226212 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:30:02.228496 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:30:02.321689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-361cd787-f2c6-47a6-b1f9-df97d33186d9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:02.322224 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-361cd787-f2c6-47a6-b1f9-df97d33186d9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:02.328994 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-361cd787-f2c6-47a6-b1f9-df97d33186d9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:02.329482 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-361cd787-f2c6-47a6-b1f9-df97d33186d9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:02.331214 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-361cd787-f2c6-47a6-b1f9-df97d33186d9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:02.331214 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-361cd787-f2c6-47a6-b1f9-df97d33186d9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:02.331214 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-361cd787-f2c6-47a6-b1f9-df97d33186d9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:02.331677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-361cd787-f2c6-47a6-b1f9-df97d33186d9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:02.446027 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8c81515d-fbb5-4f57-a5e2-3391f12d1247 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.738288 Aug 21 01:30:02.452705 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 294/601] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:30:01 2022] POST /compute/v2.1/servers => generated 380 bytes in 741 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:30:02.457798 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f0ac38bd-ee05-446d-a771-80247bd50faf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:02.458438 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f0ac38bd-ee05-446d-a771-80247bd50faf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:02.490994 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-361cd787-f2c6-47a6-b1f9-df97d33186d9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:02.501201 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-361cd787-f2c6-47a6-b1f9-df97d33186d9 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1757 microversion: 2.1 time: 0.184469 Aug 21 01:30:02.513271 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 308/602] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:02 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1757 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:02.523477 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f0ac38bd-ee05-446d-a771-80247bd50faf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:02.532876 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f0ac38bd-ee05-446d-a771-80247bd50faf tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1219 microversion: 2.1 time: 0.078876 Aug 21 01:30:02.544248 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 295/603] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:02 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1219 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:03.128933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-07b3aae0-41ba-4939-9b70-7fd46bb6b90d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:03.129652 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-07b3aae0-41ba-4939-9b70-7fd46bb6b90d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:03.137642 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-07b3aae0-41ba-4939-9b70-7fd46bb6b90d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:03.138301 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-07b3aae0-41ba-4939-9b70-7fd46bb6b90d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:03.138826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-07b3aae0-41ba-4939-9b70-7fd46bb6b90d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:03.139726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-07b3aae0-41ba-4939-9b70-7fd46bb6b90d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:03.140507 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-07b3aae0-41ba-4939-9b70-7fd46bb6b90d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:03.141019 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-07b3aae0-41ba-4939-9b70-7fd46bb6b90d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:03.284123 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-07b3aae0-41ba-4939-9b70-7fd46bb6b90d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:03.293064 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-07b3aae0-41ba-4939-9b70-7fd46bb6b90d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:03.294579 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-07b3aae0-41ba-4939-9b70-7fd46bb6b90d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:03.295788 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-07b3aae0-41ba-4939-9b70-7fd46bb6b90d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.171825 Aug 21 01:30:03.296388 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 309/604] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:03 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:30:03.523270 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-de298c06-bb33-4e0f-ba2e-8240f8b86a80 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:03.524085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-de298c06-bb33-4e0f-ba2e-8240f8b86a80 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:03.531025 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-de298c06-bb33-4e0f-ba2e-8240f8b86a80 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:03.531549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-de298c06-bb33-4e0f-ba2e-8240f8b86a80 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:03.532114 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-de298c06-bb33-4e0f-ba2e-8240f8b86a80 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:03.532940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-de298c06-bb33-4e0f-ba2e-8240f8b86a80 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:03.533455 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-de298c06-bb33-4e0f-ba2e-8240f8b86a80 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:03.533880 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-de298c06-bb33-4e0f-ba2e-8240f8b86a80 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:03.557269 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c1624c34-8eb5-4b09-827c-299094f1e81c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:03.557269 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c1624c34-8eb5-4b09-827c-299094f1e81c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:03.563062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c1624c34-8eb5-4b09-827c-299094f1e81c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:03.563659 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c1624c34-8eb5-4b09-827c-299094f1e81c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:03.564194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c1624c34-8eb5-4b09-827c-299094f1e81c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:03.564962 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c1624c34-8eb5-4b09-827c-299094f1e81c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:03.565668 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c1624c34-8eb5-4b09-827c-299094f1e81c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:03.566256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c1624c34-8eb5-4b09-827c-299094f1e81c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:03.658982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c1624c34-8eb5-4b09-827c-299094f1e81c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:03.667946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c1624c34-8eb5-4b09-827c-299094f1e81c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1223 microversion: 2.1 time: 0.116737 Aug 21 01:30:03.668565 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 310/605] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:03 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1223 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:03.817791 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-de298c06-bb33-4e0f-ba2e-8240f8b86a80 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:03.826630 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-de298c06-bb33-4e0f-ba2e-8240f8b86a80 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1760 microversion: 2.1 time: 0.308120 Aug 21 01:30:03.827265 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 296/606] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:03 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1760 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:04.319192 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b4e3de3b-5114-4dbf-801d-7c8436c767d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:04.319726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b4e3de3b-5114-4dbf-801d-7c8436c767d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:04.326425 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b4e3de3b-5114-4dbf-801d-7c8436c767d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:04.326901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b4e3de3b-5114-4dbf-801d-7c8436c767d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:04.327335 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b4e3de3b-5114-4dbf-801d-7c8436c767d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:04.328113 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b4e3de3b-5114-4dbf-801d-7c8436c767d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:04.328575 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b4e3de3b-5114-4dbf-801d-7c8436c767d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:04.329003 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b4e3de3b-5114-4dbf-801d-7c8436c767d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:04.569933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b4e3de3b-5114-4dbf-801d-7c8436c767d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:04.569933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b4e3de3b-5114-4dbf-801d-7c8436c767d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:04.571145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b4e3de3b-5114-4dbf-801d-7c8436c767d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:04.571145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b4e3de3b-5114-4dbf-801d-7c8436c767d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1722 microversion: 2.20 time: 0.157909 Aug 21 01:30:04.571145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 311/607] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:04 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1722 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:30:04.688560 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b347cd73-5bdc-430c-ba9b-bbbdee51be22 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:04.689172 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-b347cd73-5bdc-430c-ba9b-bbbdee51be22 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:04.691626 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=61,nova_api:INSERT=3 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:04.697385 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b347cd73-5bdc-430c-ba9b-bbbdee51be22 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:04.697872 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b347cd73-5bdc-430c-ba9b-bbbdee51be22 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:04.698507 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b347cd73-5bdc-430c-ba9b-bbbdee51be22 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:04.699210 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b347cd73-5bdc-430c-ba9b-bbbdee51be22 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:04.699667 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b347cd73-5bdc-430c-ba9b-bbbdee51be22 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:04.700091 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b347cd73-5bdc-430c-ba9b-bbbdee51be22 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:04.773227 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b347cd73-5bdc-430c-ba9b-bbbdee51be22 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:04.781204 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b347cd73-5bdc-430c-ba9b-bbbdee51be22 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1271 microversion: 2.1 time: 0.096480 Aug 21 01:30:04.782051 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 297/608] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:04 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1271 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:04.868284 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-74d4d13d-cc0a-40c5-b777-5111e48c14ec req-25214ed5-bb37-4f80-b8f1-9019a0998a66 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1", "name": "network-vif-deleted", "tag": "9a09b4f3-32e5-4362-82ef-ab73b634b1bf"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:04.879644 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-74d4d13d-cc0a-40c5-b777-5111e48c14ec req-25214ed5-bb37-4f80-b8f1-9019a0998a66 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:04.880205 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-74d4d13d-cc0a-40c5-b777-5111e48c14ec req-25214ed5-bb37-4f80-b8f1-9019a0998a66 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:04.880665 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-74d4d13d-cc0a-40c5-b777-5111e48c14ec req-25214ed5-bb37-4f80-b8f1-9019a0998a66 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:04.894064 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-74d4d13d-cc0a-40c5-b777-5111e48c14ec req-25214ed5-bb37-4f80-b8f1-9019a0998a66 service nova] Creating event network-vif-deleted:9a09b4f3-32e5-4362-82ef-ab73b634b1bf for instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 on hv-cinder-maste Aug 21 01:30:04.900318 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-74d4d13d-cc0a-40c5-b777-5111e48c14ec req-25214ed5-bb37-4f80-b8f1-9019a0998a66 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036074 Aug 21 01:30:04.900979 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 312/609] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:05.499416 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:05.500848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:05.509834 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:05.511143 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:05.511143 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:05.511589 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:05.512049 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:05.512498 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:05.546723 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:05.547479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:05.547909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:05.548555 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lazy-loading 'fault' on Instance uuid 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:30:05.595351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:05.605632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:05.608691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:05.608691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-65b77124-c4d1-48ba-bc95-fbea745f4698 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 200 len: 1370 microversion: 2.20 time: 0.113796 Aug 21 01:30:05.609455 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 298/610] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:05 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 1370 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:30:05.806895 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6dc10731-d705-488d-8db8-d84c809a1200 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:05.808365 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6dc10731-d705-488d-8db8-d84c809a1200 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:05.826225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6dc10731-d705-488d-8db8-d84c809a1200 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:05.827417 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6dc10731-d705-488d-8db8-d84c809a1200 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:05.828909 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6dc10731-d705-488d-8db8-d84c809a1200 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:05.839221 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6dc10731-d705-488d-8db8-d84c809a1200 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:05.839221 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6dc10731-d705-488d-8db8-d84c809a1200 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:05.839221 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6dc10731-d705-488d-8db8-d84c809a1200 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:05.934808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6dc10731-d705-488d-8db8-d84c809a1200 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:05.949490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6dc10731-d705-488d-8db8-d84c809a1200 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1277 microversion: 2.1 time: 0.150339 Aug 21 01:30:05.951782 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 313/611] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:05 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1277 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:06.632386 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d12310b3-a702-430e-8b31-a7495365159c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:06.633009 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d12310b3-a702-430e-8b31-a7495365159c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:06.640649 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d12310b3-a702-430e-8b31-a7495365159c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:06.641299 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d12310b3-a702-430e-8b31-a7495365159c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:06.641741 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d12310b3-a702-430e-8b31-a7495365159c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:06.642457 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d12310b3-a702-430e-8b31-a7495365159c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:06.642904 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d12310b3-a702-430e-8b31-a7495365159c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:06.643331 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d12310b3-a702-430e-8b31-a7495365159c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:06.658729 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-d12310b3-a702-430e-8b31-a7495365159c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] HTTP exception thrown: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d12310b3-a702-430e-8b31-a7495365159c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Request method failure captured: Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 HTTP/1.1 Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYl-urYu2WLpkg5KrWqi8t_1VSj0UQHumB0Ida0LouNet0l3Iiuz9dGUg5N8KGW-c2ywdymoBqXU-bzK3wT0eSO35h49EIb-38apkB4ujX4cihQ9PRiPI78eT9VuEcB3HW4C8TNVfy-LXwDEz2Bxg2owaQkc9JnefnISkKvong0WKOk Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Openstack-Nova-Api-Version: 2.20 Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 13b534c485db48d8a37e024702f87092 Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:30:06.660469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:30:06.662187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/13b534c485db48d8a37e024702f87092"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_13b534c485db48d8a37e024702f87092"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/13b534c485db48d8a37e024702f87092"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/13b534c485db48d8a37e024702f87092"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:30:06.662187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:30:06.662187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 13b534c485db48d8a37e024702f87092 Aug 21 01:30:06.662187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:30:06.662187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeShelveTestJSON-1747916427-project Aug 21 01:30:06.662187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:30:06.662187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:30:06.662187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: 0f4a35c46e4741c1b238f24d203b416d Aug 21 01:30:06.662187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1747916427-project Aug 21 01:30:06.662187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:30:06.662187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:30:06.662187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1'} Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:30:06.663222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:30:06.665090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:06.666349 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d12310b3-a702-430e-8b31-a7495365159c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Returning 404 to user: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:30:06.666349 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d12310b3-a702-430e-8b31-a7495365159c tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 404 len: 111 microversion: 2.20 time: 0.033993 Aug 21 01:30:06.666349 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 299/612] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:06 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 21 01:30:06.672493 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-89e92373-a64a-43c9-ab31-801d3fbd3a2f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '.version_select at 0x7f9da36865e0>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:06.750432 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-89e92373-a64a-43c9-ab31-801d3fbd3a2f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-21967027" status: 204 len: 0 microversion: 2.20 time: 0.082302 Aug 21 01:30:06.754400 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 314/613] 10.222.0.47 () {60 vars in 1256 bytes} [Sun Aug 21 01:30:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-21967027 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 21 01:30:06.979876 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c05c748f-4c90-4630-836c-44b1254eb070 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:06.980547 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c05c748f-4c90-4630-836c-44b1254eb070 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:06.986690 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c05c748f-4c90-4630-836c-44b1254eb070 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:06.987383 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c05c748f-4c90-4630-836c-44b1254eb070 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:06.987927 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c05c748f-4c90-4630-836c-44b1254eb070 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:06.988582 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c05c748f-4c90-4630-836c-44b1254eb070 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:06.989032 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c05c748f-4c90-4630-836c-44b1254eb070 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:06.989454 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c05c748f-4c90-4630-836c-44b1254eb070 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:07.400074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c05c748f-4c90-4630-836c-44b1254eb070 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:07.407966 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c05c748f-4c90-4630-836c-44b1254eb070 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1319 microversion: 2.1 time: 0.432527 Aug 21 01:30:07.408562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 300/614] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:06 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1319 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:07.714482 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-54903211-3678-4ee4-bf0d-98f136830d21 req-32530bec-5fb1-406b-aa87-78970b546a1d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad", "tag": "240f3a1c-670e-4e79-90ed-faa414cffbc0"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:07.726241 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-54903211-3678-4ee4-bf0d-98f136830d21 req-32530bec-5fb1-406b-aa87-78970b546a1d service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:07.726748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-54903211-3678-4ee4-bf0d-98f136830d21 req-32530bec-5fb1-406b-aa87-78970b546a1d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:07.727206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-54903211-3678-4ee4-bf0d-98f136830d21 req-32530bec-5fb1-406b-aa87-78970b546a1d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:07.738364 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-54903211-3678-4ee4-bf0d-98f136830d21 req-32530bec-5fb1-406b-aa87-78970b546a1d service nova] Creating event network-changed:240f3a1c-670e-4e79-90ed-faa414cffbc0 for instance bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad on hv-cinder-maste Aug 21 01:30:07.743294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-54903211-3678-4ee4-bf0d-98f136830d21 req-32530bec-5fb1-406b-aa87-78970b546a1d service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034009 Aug 21 01:30:07.744192 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 315/615] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:08.187210 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a5c4f7f4-4c16-48bd-aca5-b795851be960 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Action: 'create', calling method: .version_select at 0x7f9da33f8ca0>, body: {"keypair": {"name": "tempest-keypair-338112113", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNV6qLX3LPY9za81nXH78Tjf+/CmrK9TfFpAxJzwlIwQboVHBzTBvdium78kx95jx1ZGnSkf+5zqkoqrbWxbSRKr+SB7yx9BJ7loGOy/qhabHx7BAVEnxdl/LrkJJbYICg=="}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:08.204257 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-a5c4f7f4-4c16-48bd-aca5-b795851be960 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Getting quotas for project 13b534c485db48d8a37e024702f87092. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:30:08.208093 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-a5c4f7f4-4c16-48bd-aca5-b795851be960 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Getting quotas for user 0f4a35c46e4741c1b238f24d203b416d and project 13b534c485db48d8a37e024702f87092. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:30:08.291756 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a5c4f7f4-4c16-48bd-aca5-b795851be960 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.108139 Aug 21 01:30:08.292588 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 301/616] 10.222.0.47 () {62 vars in 1200 bytes} [Sun Aug 21 01:30:08 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 110 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Aug 21 01:30:08.431074 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-065f4aad-6a2d-41f2-93f6-746a7b5ecd8d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:08.432468 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-065f4aad-6a2d-41f2-93f6-746a7b5ecd8d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:08.439001 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-065f4aad-6a2d-41f2-93f6-746a7b5ecd8d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:08.439492 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-065f4aad-6a2d-41f2-93f6-746a7b5ecd8d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:08.439931 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-065f4aad-6a2d-41f2-93f6-746a7b5ecd8d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:08.440708 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-065f4aad-6a2d-41f2-93f6-746a7b5ecd8d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:08.441282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-065f4aad-6a2d-41f2-93f6-746a7b5ecd8d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:08.441732 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-065f4aad-6a2d-41f2-93f6-746a7b5ecd8d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:08.706910 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-065f4aad-6a2d-41f2-93f6-746a7b5ecd8d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:08.720602 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-065f4aad-6a2d-41f2-93f6-746a7b5ecd8d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1319 microversion: 2.1 time: 0.289867 Aug 21 01:30:08.720602 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 316/617] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:08 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1319 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:09.122635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bdfe3d40-98b7-4d1a-ad34-af649f3c3124 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:09.125207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bdfe3d40-98b7-4d1a-ad34-af649f3c3124 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:09.131892 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bdfe3d40-98b7-4d1a-ad34-af649f3c3124 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:09.132400 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bdfe3d40-98b7-4d1a-ad34-af649f3c3124 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:09.134351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bdfe3d40-98b7-4d1a-ad34-af649f3c3124 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:09.134351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bdfe3d40-98b7-4d1a-ad34-af649f3c3124 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:09.134558 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bdfe3d40-98b7-4d1a-ad34-af649f3c3124 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:09.134963 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bdfe3d40-98b7-4d1a-ad34-af649f3c3124 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:09.163594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bdfe3d40-98b7-4d1a-ad34-af649f3c3124 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044909 Aug 21 01:30:09.164175 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 302/618] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:30:09 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 21 01:30:09.227560 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d1fd64b6-85ef-4886-82c3-044321eb5140 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:09.228066 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d1fd64b6-85ef-4886-82c3-044321eb5140 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:09.234630 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1fd64b6-85ef-4886-82c3-044321eb5140 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:09.235117 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1fd64b6-85ef-4886-82c3-044321eb5140 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:09.235554 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1fd64b6-85ef-4886-82c3-044321eb5140 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:09.236287 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1fd64b6-85ef-4886-82c3-044321eb5140 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:09.236797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1fd64b6-85ef-4886-82c3-044321eb5140 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:09.237234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d1fd64b6-85ef-4886-82c3-044321eb5140 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:09.263348 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d1fd64b6-85ef-4886-82c3-044321eb5140 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Going to try to terminate instance {{(pid=121277) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:30:09.617596 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d1fd64b6-85ef-4886-82c3-044321eb5140 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "DELETE /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 204 len: 0 microversion: 2.1 time: 0.394055 Aug 21 01:30:09.618191 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 317/619] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:30:09 2022] DELETE /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:30:09.626616 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4ebebd26-80c3-412f-a992-7fb13f35c30a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:09.627254 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4ebebd26-80c3-412f-a992-7fb13f35c30a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:09.633813 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ebebd26-80c3-412f-a992-7fb13f35c30a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:09.634294 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ebebd26-80c3-412f-a992-7fb13f35c30a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:09.634780 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ebebd26-80c3-412f-a992-7fb13f35c30a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:09.635474 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ebebd26-80c3-412f-a992-7fb13f35c30a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:09.635928 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ebebd26-80c3-412f-a992-7fb13f35c30a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:09.636380 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ebebd26-80c3-412f-a992-7fb13f35c30a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:09.736918 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9a4afca8-4d1e-4e3f-a469-d6a921e247d7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:09.737710 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9a4afca8-4d1e-4e3f-a469-d6a921e247d7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:09.743979 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9a4afca8-4d1e-4e3f-a469-d6a921e247d7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:09.744503 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9a4afca8-4d1e-4e3f-a469-d6a921e247d7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:09.744940 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9a4afca8-4d1e-4e3f-a469-d6a921e247d7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:09.745642 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9a4afca8-4d1e-4e3f-a469-d6a921e247d7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:09.746541 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9a4afca8-4d1e-4e3f-a469-d6a921e247d7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:09.746541 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9a4afca8-4d1e-4e3f-a469-d6a921e247d7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:09.771951 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4ebebd26-80c3-412f-a992-7fb13f35c30a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:09.780293 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4ebebd26-80c3-412f-a992-7fb13f35c30a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1675 microversion: 2.1 time: 0.157769 Aug 21 01:30:09.781068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 303/620] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:09 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1675 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:09.880604 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9a4afca8-4d1e-4e3f-a469-d6a921e247d7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:09.889970 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9a4afca8-4d1e-4e3f-a469-d6a921e247d7 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1319 microversion: 2.1 time: 0.155593 Aug 21 01:30:09.889970 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 318/621] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:09 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1319 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:10.802290 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d12f3d54-a104-405a-abb0-4bf095596932 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:10.802842 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d12f3d54-a104-405a-abb0-4bf095596932 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:10.809922 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d12f3d54-a104-405a-abb0-4bf095596932 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:10.811486 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d12f3d54-a104-405a-abb0-4bf095596932 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:10.811602 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d12f3d54-a104-405a-abb0-4bf095596932 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:10.812408 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d12f3d54-a104-405a-abb0-4bf095596932 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:10.812984 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d12f3d54-a104-405a-abb0-4bf095596932 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:10.813375 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d12f3d54-a104-405a-abb0-4bf095596932 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:10.834325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:10.909747 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5653f28c-1388-4a1f-8979-321e381f1fb8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:10.910342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5653f28c-1388-4a1f-8979-321e381f1fb8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:10.917352 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5653f28c-1388-4a1f-8979-321e381f1fb8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:10.917866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5653f28c-1388-4a1f-8979-321e381f1fb8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:10.918298 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5653f28c-1388-4a1f-8979-321e381f1fb8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:10.918988 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5653f28c-1388-4a1f-8979-321e381f1fb8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:10.919439 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5653f28c-1388-4a1f-8979-321e381f1fb8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:10.919875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5653f28c-1388-4a1f-8979-321e381f1fb8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:10.948704 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d12f3d54-a104-405a-abb0-4bf095596932 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:10.957683 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d12f3d54-a104-405a-abb0-4bf095596932 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1675 microversion: 2.1 time: 0.159389 Aug 21 01:30:10.963297 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 304/622] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:10 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1675 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:11.054531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-779005687", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "cb373d64-899b-489f-9b02-6021edfc53f0"}], "security_groups": [{"name": "tempest-securitygroup--330375936"}], "key_name": "tempest-keypair-338112113", "user_data": "}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:11.059990 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5653f28c-1388-4a1f-8979-321e381f1fb8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:11.067980 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5653f28c-1388-4a1f-8979-321e381f1fb8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1319 microversion: 2.1 time: 0.162216 Aug 21 01:30:11.068600 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 319/623] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:10 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1319 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:11.163711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] validate_networks() for [('cb373d64-899b-489f-9b02-6021edfc53f0', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:30:11.538799 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:30:11.539257 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:30:11.539809 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:30:11.540328 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:30:11.540744 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:30:11.549752 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:11.550241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:11.550678 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:11.551431 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:11.551884 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:11.552354 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:11.569509 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Getting quotas for project 13b534c485db48d8a37e024702f87092. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:30:11.574151 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Getting quotas for user 0f4a35c46e4741c1b238f24d203b416d and project 13b534c485db48d8a37e024702f87092. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:30:11.585033 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:30:11.587344 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:30:11.745254 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-007a6f57-d5ea-40ea-b38f-f555d58f73d0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 0.695148 Aug 21 01:30:11.745909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 305/624] 10.222.0.47 () {62 vars in 1188 bytes} [Sun Aug 21 01:30:11 2022] POST /compute/v2.1/servers => generated 401 bytes in 696 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Aug 21 01:30:11.757454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e1276597-8df8-4adc-950c-cdca0fedd6fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:11.758066 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e1276597-8df8-4adc-950c-cdca0fedd6fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:11.811317 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e1276597-8df8-4adc-950c-cdca0fedd6fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:11.818885 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e1276597-8df8-4adc-950c-cdca0fedd6fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:11.821873 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e1276597-8df8-4adc-950c-cdca0fedd6fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:11.822681 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e1276597-8df8-4adc-950c-cdca0fedd6fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1280 microversion: 2.20 time: 0.069667 Aug 21 01:30:11.823226 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 320/625] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:11 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1280 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:30:11.979613 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-82a96386-a68a-4994-a1b7-2b98e9ebbdba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:11.980272 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-82a96386-a68a-4994-a1b7-2b98e9ebbdba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:11.988545 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-82a96386-a68a-4994-a1b7-2b98e9ebbdba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:11.988545 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-82a96386-a68a-4994-a1b7-2b98e9ebbdba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:11.988545 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-82a96386-a68a-4994-a1b7-2b98e9ebbdba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:11.988545 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-82a96386-a68a-4994-a1b7-2b98e9ebbdba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:11.989155 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-82a96386-a68a-4994-a1b7-2b98e9ebbdba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:11.989497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-82a96386-a68a-4994-a1b7-2b98e9ebbdba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:12.097468 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-bbe4f3be-2e55-46fe-8846-affbb0ca9bd4 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:12.098734 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-bbe4f3be-2e55-46fe-8846-affbb0ca9bd4 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:12.106291 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bbe4f3be-2e55-46fe-8846-affbb0ca9bd4 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:12.106774 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bbe4f3be-2e55-46fe-8846-affbb0ca9bd4 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:12.107210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bbe4f3be-2e55-46fe-8846-affbb0ca9bd4 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:12.107904 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bbe4f3be-2e55-46fe-8846-affbb0ca9bd4 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:12.108474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bbe4f3be-2e55-46fe-8846-affbb0ca9bd4 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:12.109228 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bbe4f3be-2e55-46fe-8846-affbb0ca9bd4 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:12.137477 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-82a96386-a68a-4994-a1b7-2b98e9ebbdba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:12.145820 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-82a96386-a68a-4994-a1b7-2b98e9ebbdba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1675 microversion: 2.1 time: 0.170741 Aug 21 01:30:12.146429 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 306/626] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:11 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1675 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:12.256372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-bbe4f3be-2e55-46fe-8846-affbb0ca9bd4 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:12.260029 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:12.265341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-bbe4f3be-2e55-46fe-8846-affbb0ca9bd4 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1319 microversion: 2.1 time: 0.172677 Aug 21 01:30:12.265916 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 321/627] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:12 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1319 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:12.844351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-5d06d9d3-953b-4889-98c5-3b7fea239df8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:12.844883 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-5d06d9d3-953b-4889-98c5-3b7fea239df8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:12.850874 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d06d9d3-953b-4889-98c5-3b7fea239df8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:12.851371 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d06d9d3-953b-4889-98c5-3b7fea239df8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:12.851815 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d06d9d3-953b-4889-98c5-3b7fea239df8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:12.853189 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d06d9d3-953b-4889-98c5-3b7fea239df8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:12.853694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d06d9d3-953b-4889-98c5-3b7fea239df8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:12.854075 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d06d9d3-953b-4889-98c5-3b7fea239df8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:12.917332 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5d06d9d3-953b-4889-98c5-3b7fea239df8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:12.925498 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5d06d9d3-953b-4889-98c5-3b7fea239df8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:12.927035 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5d06d9d3-953b-4889-98c5-3b7fea239df8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:12.928170 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5d06d9d3-953b-4889-98c5-3b7fea239df8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1284 microversion: 2.20 time: 0.087891 Aug 21 01:30:12.928740 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 307/628] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:12 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1284 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:30:13.167433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ae04eafa-108c-4ce2-a57d-31fecf7ab04f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:13.168079 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-ae04eafa-108c-4ce2-a57d-31fecf7ab04f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:13.176196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ae04eafa-108c-4ce2-a57d-31fecf7ab04f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:13.176748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ae04eafa-108c-4ce2-a57d-31fecf7ab04f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:13.177186 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ae04eafa-108c-4ce2-a57d-31fecf7ab04f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:13.177955 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ae04eafa-108c-4ce2-a57d-31fecf7ab04f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:13.178513 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ae04eafa-108c-4ce2-a57d-31fecf7ab04f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:13.179066 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ae04eafa-108c-4ce2-a57d-31fecf7ab04f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:13.293641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-41d2bdf3-21f8-45d7-a94b-da0af8673a4a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:13.294235 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-41d2bdf3-21f8-45d7-a94b-da0af8673a4a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:13.301275 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2bdf3-21f8-45d7-a94b-da0af8673a4a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:13.301761 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2bdf3-21f8-45d7-a94b-da0af8673a4a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:13.302197 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2bdf3-21f8-45d7-a94b-da0af8673a4a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:13.302940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2bdf3-21f8-45d7-a94b-da0af8673a4a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:13.304733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2bdf3-21f8-45d7-a94b-da0af8673a4a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:13.305166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2bdf3-21f8-45d7-a94b-da0af8673a4a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:13.314946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ae04eafa-108c-4ce2-a57d-31fecf7ab04f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:13.323635 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ae04eafa-108c-4ce2-a57d-31fecf7ab04f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1675 microversion: 2.1 time: 0.160350 Aug 21 01:30:13.324407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 322/629] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:13 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1675 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:13.517112 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-41d2bdf3-21f8-45d7-a94b-da0af8673a4a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:13.525528 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-41d2bdf3-21f8-45d7-a94b-da0af8673a4a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1319 microversion: 2.1 time: 0.235626 Aug 21 01:30:13.526449 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 308/630] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:13 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1319 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:13.950857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f549abd7-e3fd-4748-875d-84fdc4d78691 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:13.951499 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f549abd7-e3fd-4748-875d-84fdc4d78691 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:13.957953 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f549abd7-e3fd-4748-875d-84fdc4d78691 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:13.958465 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f549abd7-e3fd-4748-875d-84fdc4d78691 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:13.958903 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f549abd7-e3fd-4748-875d-84fdc4d78691 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:13.959589 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f549abd7-e3fd-4748-875d-84fdc4d78691 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:13.960124 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f549abd7-e3fd-4748-875d-84fdc4d78691 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:13.960637 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f549abd7-e3fd-4748-875d-84fdc4d78691 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:14.027664 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f549abd7-e3fd-4748-875d-84fdc4d78691 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:14.036285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f549abd7-e3fd-4748-875d-84fdc4d78691 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:14.037697 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f549abd7-e3fd-4748-875d-84fdc4d78691 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:14.038797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f549abd7-e3fd-4748-875d-84fdc4d78691 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1332 microversion: 2.20 time: 0.092339 Aug 21 01:30:14.039352 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 323/631] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:13 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1332 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:30:14.356980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c41904c2-8fa5-4a1c-a273-dfe8faa0b807 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:14.357623 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c41904c2-8fa5-4a1c-a273-dfe8faa0b807 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:14.367050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c41904c2-8fa5-4a1c-a273-dfe8faa0b807 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:14.367541 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c41904c2-8fa5-4a1c-a273-dfe8faa0b807 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:14.367684 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-1e5d3f85-2c1c-40c1-8f92-0059acdf14e8 req-0b553243-dec0-4b95-8d61-527b5d9e3ced service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c266a09a-3f81-4111-a987-b1f4be65cdc4", "name": "network-vif-deleted", "tag": "8d6fe117-8bd6-47dc-9dd9-1b92667e8ba7"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:14.369056 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c41904c2-8fa5-4a1c-a273-dfe8faa0b807 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:14.369959 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c41904c2-8fa5-4a1c-a273-dfe8faa0b807 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:14.370572 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c41904c2-8fa5-4a1c-a273-dfe8faa0b807 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:14.371157 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c41904c2-8fa5-4a1c-a273-dfe8faa0b807 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:14.378336 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-1e5d3f85-2c1c-40c1-8f92-0059acdf14e8 req-0b553243-dec0-4b95-8d61-527b5d9e3ced service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:14.378821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-1e5d3f85-2c1c-40c1-8f92-0059acdf14e8 req-0b553243-dec0-4b95-8d61-527b5d9e3ced service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:14.379254 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-1e5d3f85-2c1c-40c1-8f92-0059acdf14e8 req-0b553243-dec0-4b95-8d61-527b5d9e3ced service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:14.393203 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-1e5d3f85-2c1c-40c1-8f92-0059acdf14e8 req-0b553243-dec0-4b95-8d61-527b5d9e3ced service nova] Creating event network-vif-deleted:8d6fe117-8bd6-47dc-9dd9-1b92667e8ba7 for instance c266a09a-3f81-4111-a987-b1f4be65cdc4 on hv-cinder-maste Aug 21 01:30:14.399001 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-1e5d3f85-2c1c-40c1-8f92-0059acdf14e8 req-0b553243-dec0-4b95-8d61-527b5d9e3ced service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035707 Aug 21 01:30:14.402510 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 324/632] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:30:14.451750 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c41904c2-8fa5-4a1c-a273-dfe8faa0b807 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:14.462841 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c41904c2-8fa5-4a1c-a273-dfe8faa0b807 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1608 microversion: 2.1 time: 0.117390 Aug 21 01:30:14.479696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 309/633] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:14 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1608 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:14.546899 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a2907307-fc05-4979-8f79-951054a8a501 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:14.547758 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a2907307-fc05-4979-8f79-951054a8a501 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:14.554509 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a2907307-fc05-4979-8f79-951054a8a501 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:14.555816 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a2907307-fc05-4979-8f79-951054a8a501 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:14.556355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a2907307-fc05-4979-8f79-951054a8a501 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:14.557296 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a2907307-fc05-4979-8f79-951054a8a501 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:14.557950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a2907307-fc05-4979-8f79-951054a8a501 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:14.558586 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a2907307-fc05-4979-8f79-951054a8a501 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:14.799876 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a2907307-fc05-4979-8f79-951054a8a501 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:14.813262 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a2907307-fc05-4979-8f79-951054a8a501 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1319 microversion: 2.1 time: 0.266757 Aug 21 01:30:14.814100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 325/634] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:14 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1319 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:15.063954 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fd09c263-ea54-475e-91cf-77344100e83a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:15.064761 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fd09c263-ea54-475e-91cf-77344100e83a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:15.071253 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd09c263-ea54-475e-91cf-77344100e83a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:15.071748 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd09c263-ea54-475e-91cf-77344100e83a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:15.072347 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd09c263-ea54-475e-91cf-77344100e83a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:15.072925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd09c263-ea54-475e-91cf-77344100e83a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:15.073419 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd09c263-ea54-475e-91cf-77344100e83a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:15.073866 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd09c263-ea54-475e-91cf-77344100e83a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:15.164728 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fd09c263-ea54-475e-91cf-77344100e83a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:15.181649 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fd09c263-ea54-475e-91cf-77344100e83a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:15.183782 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fd09c263-ea54-475e-91cf-77344100e83a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:15.187228 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fd09c263-ea54-475e-91cf-77344100e83a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1340 microversion: 2.20 time: 0.128489 Aug 21 01:30:15.189393 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 310/635] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:15 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1340 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:30:15.311694 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-9a9783ce-de76-475f-ae04-70e855027c6b req-024791e7-007b-4753-8430-db6be46c7787 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad", "name": "network-vif-plugged", "status": "completed", "tag": "240f3a1c-670e-4e79-90ed-faa414cffbc0"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:15.322865 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9a9783ce-de76-475f-ae04-70e855027c6b req-024791e7-007b-4753-8430-db6be46c7787 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:15.323353 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9a9783ce-de76-475f-ae04-70e855027c6b req-024791e7-007b-4753-8430-db6be46c7787 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:15.323813 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9a9783ce-de76-475f-ae04-70e855027c6b req-024791e7-007b-4753-8430-db6be46c7787 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:15.343507 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-9a9783ce-de76-475f-ae04-70e855027c6b req-024791e7-007b-4753-8430-db6be46c7787 service nova] Creating event network-vif-plugged:240f3a1c-670e-4e79-90ed-faa414cffbc0 for instance bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad on hv-cinder-maste Aug 21 01:30:15.348950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-9a9783ce-de76-475f-ae04-70e855027c6b req-024791e7-007b-4753-8430-db6be46c7787 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041614 Aug 21 01:30:15.355178 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 326/636] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:15.650909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:15.652323 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:15.658907 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:15.659529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:15.659969 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:15.660757 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:15.661292 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:15.661725 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:15.701927 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:15.702784 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:15.703609 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:15.704331 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lazy-loading 'fault' on Instance uuid c266a09a-3f81-4111-a987-b1f4be65cdc4 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:30:15.765019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:15.776980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-41d2233a-730c-4a44-958e-167139d64ba1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 200 len: 1328 microversion: 2.1 time: 0.293473 Aug 21 01:30:15.777845 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 311/637] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:15 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 1328 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:15.839150 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-149220f2-9cec-426c-8439-97c6d39e67bb tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:15.840930 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-149220f2-9cec-426c-8439-97c6d39e67bb tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:15.847446 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-149220f2-9cec-426c-8439-97c6d39e67bb tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:15.847945 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-149220f2-9cec-426c-8439-97c6d39e67bb tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:15.848410 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-149220f2-9cec-426c-8439-97c6d39e67bb tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:15.849120 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-149220f2-9cec-426c-8439-97c6d39e67bb tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:15.849571 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-149220f2-9cec-426c-8439-97c6d39e67bb tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:15.849994 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-149220f2-9cec-426c-8439-97c6d39e67bb tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:15.942001 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:30:15.942563 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Sun Aug 21 01:30:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad (ip 10.222.0.47) !!! Aug 21 01:30:15.999596 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-149220f2-9cec-426c-8439-97c6d39e67bb tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:16.013280 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-149220f2-9cec-426c-8439-97c6d39e67bb tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1319 microversion: 2.1 time: 0.177256 Aug 21 01:30:16.013280 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 327/638] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:15 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1319 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:16.212184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bb230365-ed47-4b22-8b90-d65b1e37e726 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:16.213082 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bb230365-ed47-4b22-8b90-d65b1e37e726 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:16.219381 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bb230365-ed47-4b22-8b90-d65b1e37e726 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:16.219867 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bb230365-ed47-4b22-8b90-d65b1e37e726 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:16.220338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bb230365-ed47-4b22-8b90-d65b1e37e726 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:16.221037 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bb230365-ed47-4b22-8b90-d65b1e37e726 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:16.221522 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bb230365-ed47-4b22-8b90-d65b1e37e726 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:16.221950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bb230365-ed47-4b22-8b90-d65b1e37e726 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:16.360785 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-bb230365-ed47-4b22-8b90-d65b1e37e726 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:16.368306 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-bb230365-ed47-4b22-8b90-d65b1e37e726 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:16.369839 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-bb230365-ed47-4b22-8b90-d65b1e37e726 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:16.370997 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bb230365-ed47-4b22-8b90-d65b1e37e726 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1405 microversion: 2.20 time: 0.162849 Aug 21 01:30:16.371550 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 312/639] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:16 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1405 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:30:16.731555 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-f0bab9a2-d161-4fed-95e4-ac2f95c596e6 req-41b16408-5919-4293-aa97-cfabef5afef9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a5b475aa-486a-4957-8b54-aa1ee7438a8a", "tag": "d3beeb80-5980-4de1-b832-2d4a1e1dae1c"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:16.742219 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-f0bab9a2-d161-4fed-95e4-ac2f95c596e6 req-41b16408-5919-4293-aa97-cfabef5afef9 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:16.742693 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-f0bab9a2-d161-4fed-95e4-ac2f95c596e6 req-41b16408-5919-4293-aa97-cfabef5afef9 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:16.743119 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-f0bab9a2-d161-4fed-95e4-ac2f95c596e6 req-41b16408-5919-4293-aa97-cfabef5afef9 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:16.753444 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-f0bab9a2-d161-4fed-95e4-ac2f95c596e6 req-41b16408-5919-4293-aa97-cfabef5afef9 service nova] Creating event network-changed:d3beeb80-5980-4de1-b832-2d4a1e1dae1c for instance a5b475aa-486a-4957-8b54-aa1ee7438a8a on hv-cinder-maste Aug 21 01:30:16.758265 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-f0bab9a2-d161-4fed-95e4-ac2f95c596e6 req-41b16408-5919-4293-aa97-cfabef5afef9 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030561 Aug 21 01:30:16.759634 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 328/640] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:30:16.797788 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b8bedab4-c53a-4b73-afab-f1e45b333db1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:16.798400 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-b8bedab4-c53a-4b73-afab-f1e45b333db1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:16.805009 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b8bedab4-c53a-4b73-afab-f1e45b333db1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:16.805507 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b8bedab4-c53a-4b73-afab-f1e45b333db1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:16.805933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b8bedab4-c53a-4b73-afab-f1e45b333db1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:16.806621 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b8bedab4-c53a-4b73-afab-f1e45b333db1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:16.807076 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b8bedab4-c53a-4b73-afab-f1e45b333db1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:16.807521 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b8bedab4-c53a-4b73-afab-f1e45b333db1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:16.821917 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-b8bedab4-c53a-4b73-afab-f1e45b333db1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] HTTP exception thrown: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b8bedab4-c53a-4b73-afab-f1e45b333db1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Request method failure captured: Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 HTTP/1.1 Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYl7pq5WVSQzS51LpLVCZZUw27iHEeVZJoQRcIvRcn8IboU95JITc9pVqybD2xzZX2nmxh3OfTWCk6tel2UXiN18o1XKAXaRJqQ1AV1sQ2sNj8ZWXwdNkHZXM2QMX9i5RlP6rG_R721C3RJRTJbjQ-MvmqR7-hIA4GefX0KAYXbHx7g Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: f4a0fcea08b248fe84012875a7243761 Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:30:16.823440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:30:16.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_f4a0fcea08b248fe84012875a7243761"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:30:16.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:30:16.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: f4a0fcea08b248fe84012875a7243761 Aug 21 01:30:16.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:30:16.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeTestJSON-528571573-project Aug 21 01:30:16.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:30:16.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:30:16.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: 0cd8d822f15943a78575c3744692cdb9 Aug 21 01:30:16.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeTestJSON-528571573-project Aug 21 01:30:16.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:30:16.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:30:16.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': 'c266a09a-3f81-4111-a987-b1f4be65cdc4'} Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:30:16.825921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:30:16.827184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:16.828324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b8bedab4-c53a-4b73-afab-f1e45b333db1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Returning 404 to user: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:30:16.828324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b8bedab4-c53a-4b73-afab-f1e45b333db1 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 404 len: 111 microversion: 2.1 time: 0.031121 Aug 21 01:30:16.828324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 313/641] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:16 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:30:16.834554 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-48fc90df-4c61-4a9a-8a6a-1215c9d3ea7f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '.version_select at 0x7f9da351cb80>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:16.933665 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-48fc90df-4c61-4a9a-8a6a-1215c9d3ea7f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1692936143" status: 202 len: 0 microversion: 2.1 time: 0.103123 Aug 21 01:30:16.934322 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 329/642] 10.222.0.47 () {58 vars in 1221 bytes} [Sun Aug 21 01:30:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1692936143 => generated 0 bytes in 105 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:30:17.031689 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-04a744e3-9798-400a-8cc4-8baf93191d9c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:17.032436 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-04a744e3-9798-400a-8cc4-8baf93191d9c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:17.038507 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-04a744e3-9798-400a-8cc4-8baf93191d9c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:17.039068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-04a744e3-9798-400a-8cc4-8baf93191d9c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:17.039575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-04a744e3-9798-400a-8cc4-8baf93191d9c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:17.040363 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-04a744e3-9798-400a-8cc4-8baf93191d9c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:17.040809 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-04a744e3-9798-400a-8cc4-8baf93191d9c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:17.041226 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-04a744e3-9798-400a-8cc4-8baf93191d9c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:17.204374 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-04a744e3-9798-400a-8cc4-8baf93191d9c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:17.214574 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-04a744e3-9798-400a-8cc4-8baf93191d9c tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1319 microversion: 2.1 time: 0.186637 Aug 21 01:30:17.215904 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 314/643] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:17 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1319 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:17.242761 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a72c19e2-a134-47e6-8dbd-2688cf6d8fb8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:17.244231 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a72c19e2-a134-47e6-8dbd-2688cf6d8fb8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:17.251928 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a72c19e2-a134-47e6-8dbd-2688cf6d8fb8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:17.252714 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a72c19e2-a134-47e6-8dbd-2688cf6d8fb8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:17.253326 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a72c19e2-a134-47e6-8dbd-2688cf6d8fb8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:17.254222 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a72c19e2-a134-47e6-8dbd-2688cf6d8fb8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:17.254937 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a72c19e2-a134-47e6-8dbd-2688cf6d8fb8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:17.255602 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a72c19e2-a134-47e6-8dbd-2688cf6d8fb8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:17.396776 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-cdd19612-9c4d-4a97-944b-5f30a07d0217 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:17.397852 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-cdd19612-9c4d-4a97-944b-5f30a07d0217 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:17.406026 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cdd19612-9c4d-4a97-944b-5f30a07d0217 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:17.411130 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cdd19612-9c4d-4a97-944b-5f30a07d0217 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:17.411919 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cdd19612-9c4d-4a97-944b-5f30a07d0217 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:17.412832 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cdd19612-9c4d-4a97-944b-5f30a07d0217 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:17.413518 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cdd19612-9c4d-4a97-944b-5f30a07d0217 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:17.414188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cdd19612-9c4d-4a97-944b-5f30a07d0217 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:17.416191 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-a72c19e2-a134-47e6-8dbd-2688cf6d8fb8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lazy-loading 'services' on Instance uuid eddb6b3f-8ca0-4f71-b371-19c6e65dd095 {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:30:17.640648 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-cdd19612-9c4d-4a97-944b-5f30a07d0217 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:17.648983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-cdd19612-9c4d-4a97-944b-5f30a07d0217 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:17.650549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-cdd19612-9c4d-4a97-944b-5f30a07d0217 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:17.651653 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-cdd19612-9c4d-4a97-944b-5f30a07d0217 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1405 microversion: 2.20 time: 0.259138 Aug 21 01:30:17.661257 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 315/644] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:17 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1405 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:30:17.760678 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a72c19e2-a134-47e6-8dbd-2688cf6d8fb8 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "DELETE /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments/2fb97ee5-b984-45f9-8ae3-f2209e4b20e3" status: 202 len: 0 microversion: 2.1 time: 0.521837 Aug 21 01:30:17.761189 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 330/645] 10.222.0.47 () {58 vars in 1416 bytes} [Sun Aug 21 01:30:17 2022] DELETE /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments/2fb97ee5-b984-45f9-8ae3-f2209e4b20e3 => generated 0 bytes in 524 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:30:17.769566 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-021fd1e8-1d01-410b-978a-a5395b0e6a2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:17.772199 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-021fd1e8-1d01-410b-978a-a5395b0e6a2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:17.778635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-021fd1e8-1d01-410b-978a-a5395b0e6a2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:17.779213 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-021fd1e8-1d01-410b-978a-a5395b0e6a2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:17.779747 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-021fd1e8-1d01-410b-978a-a5395b0e6a2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:17.780539 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-021fd1e8-1d01-410b-978a-a5395b0e6a2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:17.781179 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-021fd1e8-1d01-410b-978a-a5395b0e6a2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:17.781888 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-021fd1e8-1d01-410b-978a-a5395b0e6a2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:17.821795 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-021fd1e8-1d01-410b-978a-a5395b0e6a2c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056164 Aug 21 01:30:17.823034 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 316/646] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:30:17 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:17.958801 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b39ff75c-22ac-4070-9678-6b59e878bd08 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Action: 'create', calling method: .version_select at 0x7f9da36865e0>, body: {"keypair": {"name": "tempest-keypair-899759034", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHck363Q0yeLTQNnDKiWMv2icTkcMFsMSjslDQoZEEp9pxfj1A7zhhO0Du3POo8s43aubjyyy+rN4Kfh2xzQIdvq/BOErBZIJLs+3pkP5LDdpn/QimnnNtHCULJKufVzzA=="}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:17.980827 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-b39ff75c-22ac-4070-9678-6b59e878bd08 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Getting quotas for project f4a0fcea08b248fe84012875a7243761. Resources: {'key_pairs'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:30:17.985607 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-b39ff75c-22ac-4070-9678-6b59e878bd08 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Getting quotas for user 0cd8d822f15943a78575c3744692cdb9 and project f4a0fcea08b248fe84012875a7243761. Resources: {'key_pairs'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:30:17.996751 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:18.137760 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b39ff75c-22ac-4070-9678-6b59e878bd08 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.184533 Aug 21 01:30:18.138467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 331/647] 10.222.0.47 () {60 vars in 1159 bytes} [Sun Aug 21 01:30:17 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:18.236796 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3fdff661-523d-4fb8-8d29-6a607776c08a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:18.237414 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3fdff661-523d-4fb8-8d29-6a607776c08a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:18.244542 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fdff661-523d-4fb8-8d29-6a607776c08a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:18.246086 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fdff661-523d-4fb8-8d29-6a607776c08a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:18.246086 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fdff661-523d-4fb8-8d29-6a607776c08a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:18.246317 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fdff661-523d-4fb8-8d29-6a607776c08a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:18.246827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fdff661-523d-4fb8-8d29-6a607776c08a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:18.247257 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3fdff661-523d-4fb8-8d29-6a607776c08a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:18.435811 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3fdff661-523d-4fb8-8d29-6a607776c08a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:18.445562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3fdff661-523d-4fb8-8d29-6a607776c08a tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1508 microversion: 2.1 time: 0.212870 Aug 21 01:30:18.446141 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 317/648] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:18 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1508 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:18.471266 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-aecba68c-02af-4e8d-a305-7298132161b8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "fdcfe337-2074-4bbd-bad1-862c9ae74f17", "device": "/dev/sdb"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:18.474771 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-aecba68c-02af-4e8d-a305-7298132161b8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:18.488016 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aecba68c-02af-4e8d-a305-7298132161b8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:18.488567 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aecba68c-02af-4e8d-a305-7298132161b8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:18.489224 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aecba68c-02af-4e8d-a305-7298132161b8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:18.490758 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aecba68c-02af-4e8d-a305-7298132161b8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:18.491213 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aecba68c-02af-4e8d-a305-7298132161b8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:18.491654 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aecba68c-02af-4e8d-a305-7298132161b8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:18.735988 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8df389f0-6c5a-410f-a78d-74eaf9512acd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:18.736633 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8df389f0-6c5a-410f-a78d-74eaf9512acd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:18.743285 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8df389f0-6c5a-410f-a78d-74eaf9512acd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:18.743772 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8df389f0-6c5a-410f-a78d-74eaf9512acd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:18.744472 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8df389f0-6c5a-410f-a78d-74eaf9512acd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:18.745168 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8df389f0-6c5a-410f-a78d-74eaf9512acd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:18.745820 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8df389f0-6c5a-410f-a78d-74eaf9512acd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:18.746249 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8df389f0-6c5a-410f-a78d-74eaf9512acd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:19.218675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8df389f0-6c5a-410f-a78d-74eaf9512acd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:19.227607 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8df389f0-6c5a-410f-a78d-74eaf9512acd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:19.229163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8df389f0-6c5a-410f-a78d-74eaf9512acd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:19.230629 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8df389f0-6c5a-410f-a78d-74eaf9512acd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1405 microversion: 2.20 time: 0.559305 Aug 21 01:30:19.231202 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 318/649] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:18 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1405 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:30:19.235968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a4d79b16-0c6a-4a2d-b0da-0f38e8a459ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:19.238290 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a4d79b16-0c6a-4a2d-b0da-0f38e8a459ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:19.244904 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4d79b16-0c6a-4a2d-b0da-0f38e8a459ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:19.245398 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4d79b16-0c6a-4a2d-b0da-0f38e8a459ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:19.245839 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4d79b16-0c6a-4a2d-b0da-0f38e8a459ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:19.246557 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4d79b16-0c6a-4a2d-b0da-0f38e8a459ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:19.247011 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4d79b16-0c6a-4a2d-b0da-0f38e8a459ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:19.247501 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4d79b16-0c6a-4a2d-b0da-0f38e8a459ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:19.281468 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a4d79b16-0c6a-4a2d-b0da-0f38e8a459ff tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049048 Aug 21 01:30:19.282042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 319/650] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:30:19 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:19.728233 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-aecba68c-02af-4e8d-a305-7298132161b8 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "POST /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.260766 Aug 21 01:30:19.734999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 332/651] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:30:18 2022] POST /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad/os-volume_attachments => generated 194 bytes in 1263 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:20.252634 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1eae8d1a-221a-4ca3-807f-69ee42d920dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:20.253336 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1eae8d1a-221a-4ca3-807f-69ee42d920dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:20.259936 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eae8d1a-221a-4ca3-807f-69ee42d920dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:20.260472 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eae8d1a-221a-4ca3-807f-69ee42d920dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:20.260906 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eae8d1a-221a-4ca3-807f-69ee42d920dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:20.261596 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eae8d1a-221a-4ca3-807f-69ee42d920dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:20.262072 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eae8d1a-221a-4ca3-807f-69ee42d920dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:20.262489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1eae8d1a-221a-4ca3-807f-69ee42d920dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:20.296353 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8c911cc5-47d3-4aa9-9610-d822fb76e605 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:20.298864 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8c911cc5-47d3-4aa9-9610-d822fb76e605 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:20.305911 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c911cc5-47d3-4aa9-9610-d822fb76e605 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:20.306407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c911cc5-47d3-4aa9-9610-d822fb76e605 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:20.306844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c911cc5-47d3-4aa9-9610-d822fb76e605 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:20.307546 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c911cc5-47d3-4aa9-9610-d822fb76e605 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:20.307995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c911cc5-47d3-4aa9-9610-d822fb76e605 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:20.308449 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c911cc5-47d3-4aa9-9610-d822fb76e605 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:20.343901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8c911cc5-47d3-4aa9-9610-d822fb76e605 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052439 Aug 21 01:30:20.344930 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 333/652] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:30:20 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:30:20.408847 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1eae8d1a-221a-4ca3-807f-69ee42d920dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:20.418123 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1eae8d1a-221a-4ca3-807f-69ee42d920dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:20.419716 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1eae8d1a-221a-4ca3-807f-69ee42d920dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:20.420887 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1eae8d1a-221a-4ca3-807f-69ee42d920dc tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1405 microversion: 2.20 time: 0.172239 Aug 21 01:30:20.421660 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 320/653] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:20 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1405 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:30:21.158004 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-2023500478", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "9ee81273-c1c2-4ebe-9984-21786c936dee"}], "security_groups": [{"name": "tempest-securitygroup--8484626"}], "key_name": "tempest-keypair-899759034", "user_data": "}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:21.263404 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.network.neutron [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] validate_networks() for [('9ee81273-c1c2-4ebe-9984-21786c936dee', None, None, None, None, None)] {{(pid=121277) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:30:21.356611 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3e3f9501-8648-49dd-a32f-f6d820debbd0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:21.358806 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3e3f9501-8648-49dd-a32f-f6d820debbd0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:21.365441 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3e3f9501-8648-49dd-a32f-f6d820debbd0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:21.365918 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3e3f9501-8648-49dd-a32f-f6d820debbd0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:21.366398 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3e3f9501-8648-49dd-a32f-f6d820debbd0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:21.367244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3e3f9501-8648-49dd-a32f-f6d820debbd0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:21.367695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3e3f9501-8648-49dd-a32f-f6d820debbd0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:21.368147 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3e3f9501-8648-49dd-a32f-f6d820debbd0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:21.399924 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3e3f9501-8648-49dd-a32f-f6d820debbd0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047790 Aug 21 01:30:21.407130 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 321/654] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:30:21 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Aug 21 01:30:21.442921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-44b303ce-a585-4c84-9b15-e238690afdd3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:21.443709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-44b303ce-a585-4c84-9b15-e238690afdd3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:21.456775 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-44b303ce-a585-4c84-9b15-e238690afdd3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:21.456775 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-44b303ce-a585-4c84-9b15-e238690afdd3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:21.456775 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-44b303ce-a585-4c84-9b15-e238690afdd3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:21.456775 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-44b303ce-a585-4c84-9b15-e238690afdd3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:21.457221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-44b303ce-a585-4c84-9b15-e238690afdd3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:21.457221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-44b303ce-a585-4c84-9b15-e238690afdd3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:21.463707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=100 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:21.560088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=4 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:21.581074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=18 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:21.672058 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-44b303ce-a585-4c84-9b15-e238690afdd3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:21.684869 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-44b303ce-a585-4c84-9b15-e238690afdd3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:21.686399 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-44b303ce-a585-4c84-9b15-e238690afdd3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:21.687595 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-44b303ce-a585-4c84-9b15-e238690afdd3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1405 microversion: 2.20 time: 0.248710 Aug 21 01:30:21.688261 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 322/655] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:21 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1405 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:30:21.709468 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ab4cdcbf-ad17-4c33-a139-a7008a54f1bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:21.709975 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ab4cdcbf-ad17-4c33-a139-a7008a54f1bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:21.716701 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ab4cdcbf-ad17-4c33-a139-a7008a54f1bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:21.717198 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ab4cdcbf-ad17-4c33-a139-a7008a54f1bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:21.717640 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ab4cdcbf-ad17-4c33-a139-a7008a54f1bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:21.718372 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ab4cdcbf-ad17-4c33-a139-a7008a54f1bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:21.718832 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ab4cdcbf-ad17-4c33-a139-a7008a54f1bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:21.719281 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ab4cdcbf-ad17-4c33-a139-a7008a54f1bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:21.747618 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ab4cdcbf-ad17-4c33-a139-a7008a54f1bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:30:21.772158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Flavor limits 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:30:21.772589 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Image limits 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:30:21.773063 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Flavor pref 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:30:21.773522 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Image pref 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:30:21.773995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:30:21.786565 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:21.787070 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:21.787544 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:21.788360 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:21.788853 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:21.789281 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:21.806803 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Getting quotas for project f4a0fcea08b248fe84012875a7243761. Resources: {'instances', 'ram', 'cores'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:30:21.810927 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Getting quotas for user 0cd8d822f15943a78575c3744692cdb9 and project f4a0fcea08b248fe84012875a7243761. Resources: {'instances', 'ram', 'cores'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:30:21.823239 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Going to run 1 instances... {{(pid=121277) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:30:21.825703 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121277) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:30:21.994267 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f055dbe0-7293-4d72-b2ed-cf0916395a7c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 399 microversion: 2.1 time: 0.841692 Aug 21 01:30:21.994783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 334/656] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:30:21 2022] POST /compute/v2.1/servers => generated 399 bytes in 844 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:30:22.010662 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-60413f23-c8a3-4e6b-a8a8-81d4ce372bd8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:22.012953 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-60413f23-c8a3-4e6b-a8a8-81d4ce372bd8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:22.075065 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-60413f23-c8a3-4e6b-a8a8-81d4ce372bd8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:22.084880 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-60413f23-c8a3-4e6b-a8a8-81d4ce372bd8 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1237 microversion: 2.1 time: 0.078838 Aug 21 01:30:22.093882 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 335/657] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:22 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1237 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:22.114159 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ab4cdcbf-ad17-4c33-a139-a7008a54f1bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "DELETE /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 204 len: 0 microversion: 2.1 time: 0.412374 Aug 21 01:30:22.114810 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 323/658] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:30:21 2022] DELETE /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:30:22.128708 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-79988962-6f01-481d-afee-4b96cc3be8c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:22.129353 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-79988962-6f01-481d-afee-4b96cc3be8c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:22.135587 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79988962-6f01-481d-afee-4b96cc3be8c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:22.136068 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79988962-6f01-481d-afee-4b96cc3be8c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:22.136559 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79988962-6f01-481d-afee-4b96cc3be8c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:22.137232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79988962-6f01-481d-afee-4b96cc3be8c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:22.137677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79988962-6f01-481d-afee-4b96cc3be8c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:22.138145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79988962-6f01-481d-afee-4b96cc3be8c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:22.299825 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-79988962-6f01-481d-afee-4b96cc3be8c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:22.303266 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-79988962-6f01-481d-afee-4b96cc3be8c5 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1688 microversion: 2.1 time: 0.178549 Aug 21 01:30:22.303844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 336/659] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:22 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1688 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:22.716074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f3cd5567-5b6e-46d5-98ac-7c96e4e1f096 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:22.716814 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f3cd5567-5b6e-46d5-98ac-7c96e4e1f096 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:22.723468 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f3cd5567-5b6e-46d5-98ac-7c96e4e1f096 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:22.723986 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f3cd5567-5b6e-46d5-98ac-7c96e4e1f096 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:22.724478 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f3cd5567-5b6e-46d5-98ac-7c96e4e1f096 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:22.725195 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f3cd5567-5b6e-46d5-98ac-7c96e4e1f096 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:22.725647 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f3cd5567-5b6e-46d5-98ac-7c96e4e1f096 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:22.726078 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f3cd5567-5b6e-46d5-98ac-7c96e4e1f096 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:22.887780 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f3cd5567-5b6e-46d5-98ac-7c96e4e1f096 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:22.898240 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f3cd5567-5b6e-46d5-98ac-7c96e4e1f096 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:22.901005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f3cd5567-5b6e-46d5-98ac-7c96e4e1f096 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:22.902440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f3cd5567-5b6e-46d5-98ac-7c96e4e1f096 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1405 microversion: 2.20 time: 0.191282 Aug 21 01:30:22.903264 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 324/660] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:22 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1405 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:30:23.106353 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-34345bfd-7999-43b7-bf26-19b159557f15 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:23.107556 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-34345bfd-7999-43b7-bf26-19b159557f15 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:23.113518 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-34345bfd-7999-43b7-bf26-19b159557f15 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:23.114017 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-34345bfd-7999-43b7-bf26-19b159557f15 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:23.114459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-34345bfd-7999-43b7-bf26-19b159557f15 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:23.115171 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-34345bfd-7999-43b7-bf26-19b159557f15 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:23.115677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-34345bfd-7999-43b7-bf26-19b159557f15 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:23.116129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-34345bfd-7999-43b7-bf26-19b159557f15 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:23.189673 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-34345bfd-7999-43b7-bf26-19b159557f15 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:23.202581 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-34345bfd-7999-43b7-bf26-19b159557f15 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1241 microversion: 2.1 time: 0.100128 Aug 21 01:30:23.203299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 337/661] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:23 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1241 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:23.326584 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9ae36907-61c4-439d-b285-55b484cf75f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:23.327578 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9ae36907-61c4-439d-b285-55b484cf75f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:23.334226 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ae36907-61c4-439d-b285-55b484cf75f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:23.334651 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ae36907-61c4-439d-b285-55b484cf75f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:23.335094 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ae36907-61c4-439d-b285-55b484cf75f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:23.335800 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ae36907-61c4-439d-b285-55b484cf75f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:23.336308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ae36907-61c4-439d-b285-55b484cf75f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:23.336736 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ae36907-61c4-439d-b285-55b484cf75f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:23.475783 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9ae36907-61c4-439d-b285-55b484cf75f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:23.488536 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9ae36907-61c4-439d-b285-55b484cf75f0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1688 microversion: 2.1 time: 0.166858 Aug 21 01:30:23.489508 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 325/662] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:23 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1688 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:23.927142 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f8791c47-66c3-4cb2-bb7a-bfed1145e2d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:23.928009 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f8791c47-66c3-4cb2-bb7a-bfed1145e2d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:23.935803 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8791c47-66c3-4cb2-bb7a-bfed1145e2d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:23.936326 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8791c47-66c3-4cb2-bb7a-bfed1145e2d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:23.936849 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8791c47-66c3-4cb2-bb7a-bfed1145e2d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:23.937539 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8791c47-66c3-4cb2-bb7a-bfed1145e2d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:23.937985 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8791c47-66c3-4cb2-bb7a-bfed1145e2d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:23.938407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8791c47-66c3-4cb2-bb7a-bfed1145e2d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:24.076942 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f8791c47-66c3-4cb2-bb7a-bfed1145e2d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:24.085132 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f8791c47-66c3-4cb2-bb7a-bfed1145e2d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:24.086765 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f8791c47-66c3-4cb2-bb7a-bfed1145e2d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:24.088355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f8791c47-66c3-4cb2-bb7a-bfed1145e2d8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1405 microversion: 2.20 time: 0.165348 Aug 21 01:30:24.088917 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 338/663] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:23 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1405 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:30:24.223845 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-996ccdb5-fd0e-48f4-9a2f-0fd0a8a77515 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:24.224537 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-996ccdb5-fd0e-48f4-9a2f-0fd0a8a77515 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:24.231209 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-996ccdb5-fd0e-48f4-9a2f-0fd0a8a77515 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:24.231752 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-996ccdb5-fd0e-48f4-9a2f-0fd0a8a77515 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:24.232134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-996ccdb5-fd0e-48f4-9a2f-0fd0a8a77515 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:24.232830 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-996ccdb5-fd0e-48f4-9a2f-0fd0a8a77515 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:24.233274 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-996ccdb5-fd0e-48f4-9a2f-0fd0a8a77515 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:24.233693 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-996ccdb5-fd0e-48f4-9a2f-0fd0a8a77515 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:24.299807 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-996ccdb5-fd0e-48f4-9a2f-0fd0a8a77515 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:24.307968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-996ccdb5-fd0e-48f4-9a2f-0fd0a8a77515 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1241 microversion: 2.1 time: 0.087935 Aug 21 01:30:24.308802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 326/664] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:24 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1241 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:24.520415 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-37746b9c-a7e6-4b18-b2ac-fb2b38784785 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:24.521061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-37746b9c-a7e6-4b18-b2ac-fb2b38784785 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:24.528178 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37746b9c-a7e6-4b18-b2ac-fb2b38784785 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:24.528740 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37746b9c-a7e6-4b18-b2ac-fb2b38784785 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:24.529185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37746b9c-a7e6-4b18-b2ac-fb2b38784785 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:24.529867 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37746b9c-a7e6-4b18-b2ac-fb2b38784785 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:24.530323 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37746b9c-a7e6-4b18-b2ac-fb2b38784785 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:24.530768 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-37746b9c-a7e6-4b18-b2ac-fb2b38784785 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:24.671783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-37746b9c-a7e6-4b18-b2ac-fb2b38784785 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:24.679637 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-37746b9c-a7e6-4b18-b2ac-fb2b38784785 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1688 microversion: 2.1 time: 0.163280 Aug 21 01:30:24.680385 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 339/665] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:24 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1688 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:25.091936 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-2a3ae262-9f98-4653-a9bf-67aef796ab84 req-5945209c-1049-4801-9a2b-eccd38c8969d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a5b475aa-486a-4957-8b54-aa1ee7438a8a", "name": "network-vif-plugged", "status": "completed", "tag": "d3beeb80-5980-4de1-b832-2d4a1e1dae1c"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:25.101997 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2a3ae262-9f98-4653-a9bf-67aef796ab84 req-5945209c-1049-4801-9a2b-eccd38c8969d service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:25.102506 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2a3ae262-9f98-4653-a9bf-67aef796ab84 req-5945209c-1049-4801-9a2b-eccd38c8969d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:25.103504 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-2a3ae262-9f98-4653-a9bf-67aef796ab84 req-5945209c-1049-4801-9a2b-eccd38c8969d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:25.111044 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-951f1592-babd-4ebf-a185-ae3fabfc8bd1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:25.111690 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-951f1592-babd-4ebf-a185-ae3fabfc8bd1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:25.116613 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-2a3ae262-9f98-4653-a9bf-67aef796ab84 req-5945209c-1049-4801-9a2b-eccd38c8969d service nova] Creating event network-vif-plugged:d3beeb80-5980-4de1-b832-2d4a1e1dae1c for instance a5b475aa-486a-4957-8b54-aa1ee7438a8a on hv-cinder-maste Aug 21 01:30:25.126275 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-951f1592-babd-4ebf-a185-ae3fabfc8bd1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:25.126275 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-951f1592-babd-4ebf-a185-ae3fabfc8bd1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:25.126275 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-951f1592-babd-4ebf-a185-ae3fabfc8bd1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:25.127020 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-2a3ae262-9f98-4653-a9bf-67aef796ab84 req-5945209c-1049-4801-9a2b-eccd38c8969d service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034689 Aug 21 01:30:25.127121 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-951f1592-babd-4ebf-a185-ae3fabfc8bd1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:25.127121 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-951f1592-babd-4ebf-a185-ae3fabfc8bd1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:25.127289 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 327/666] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:25.127407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-951f1592-babd-4ebf-a185-ae3fabfc8bd1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:25.260554 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-951f1592-babd-4ebf-a185-ae3fabfc8bd1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:25.269484 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-951f1592-babd-4ebf-a185-ae3fabfc8bd1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:25.271078 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-951f1592-babd-4ebf-a185-ae3fabfc8bd1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:25.272225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-951f1592-babd-4ebf-a185-ae3fabfc8bd1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1405 microversion: 2.20 time: 0.165141 Aug 21 01:30:25.272765 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 340/667] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:25 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1405 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:30:25.329864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-aecf01e3-3e55-41e0-96d4-d5a7bd337248 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:25.330703 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-aecf01e3-3e55-41e0-96d4-d5a7bd337248 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:25.337007 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aecf01e3-3e55-41e0-96d4-d5a7bd337248 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:25.338054 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aecf01e3-3e55-41e0-96d4-d5a7bd337248 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:25.338503 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aecf01e3-3e55-41e0-96d4-d5a7bd337248 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:25.339240 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aecf01e3-3e55-41e0-96d4-d5a7bd337248 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:25.339700 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aecf01e3-3e55-41e0-96d4-d5a7bd337248 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:25.340225 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aecf01e3-3e55-41e0-96d4-d5a7bd337248 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:25.404986 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-aecf01e3-3e55-41e0-96d4-d5a7bd337248 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:25.412763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-aecf01e3-3e55-41e0-96d4-d5a7bd337248 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1241 microversion: 2.1 time: 0.087556 Aug 21 01:30:25.413356 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 328/668] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:25 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1241 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:25.701644 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-dd8e1184-1c2b-4cb6-8465-e35d717466ce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:25.702157 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-dd8e1184-1c2b-4cb6-8465-e35d717466ce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:25.709139 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dd8e1184-1c2b-4cb6-8465-e35d717466ce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:25.709615 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dd8e1184-1c2b-4cb6-8465-e35d717466ce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:25.710093 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dd8e1184-1c2b-4cb6-8465-e35d717466ce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:25.710796 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dd8e1184-1c2b-4cb6-8465-e35d717466ce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:25.711245 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dd8e1184-1c2b-4cb6-8465-e35d717466ce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:25.711669 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dd8e1184-1c2b-4cb6-8465-e35d717466ce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:25.850029 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-dd8e1184-1c2b-4cb6-8465-e35d717466ce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:25.857632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-dd8e1184-1c2b-4cb6-8465-e35d717466ce tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1688 microversion: 2.1 time: 0.160118 Aug 21 01:30:25.858200 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 341/669] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:25 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1688 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:26.297948 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1aeb1dcc-f276-47b1-b48d-7512582d8930 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:26.298950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1aeb1dcc-f276-47b1-b48d-7512582d8930 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:26.301050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:26.305962 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1aeb1dcc-f276-47b1-b48d-7512582d8930 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:26.306458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1aeb1dcc-f276-47b1-b48d-7512582d8930 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:26.309060 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1aeb1dcc-f276-47b1-b48d-7512582d8930 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:26.309060 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1aeb1dcc-f276-47b1-b48d-7512582d8930 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:26.309060 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1aeb1dcc-f276-47b1-b48d-7512582d8930 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:26.309060 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1aeb1dcc-f276-47b1-b48d-7512582d8930 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:26.434564 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-755d6f04-3439-4f9f-9ac4-a7ccf55f79ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:26.435886 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-755d6f04-3439-4f9f-9ac4-a7ccf55f79ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:26.442880 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-755d6f04-3439-4f9f-9ac4-a7ccf55f79ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:26.443467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-755d6f04-3439-4f9f-9ac4-a7ccf55f79ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:26.443984 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-755d6f04-3439-4f9f-9ac4-a7ccf55f79ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:26.444818 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-755d6f04-3439-4f9f-9ac4-a7ccf55f79ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:26.445379 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-755d6f04-3439-4f9f-9ac4-a7ccf55f79ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:26.446155 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-755d6f04-3439-4f9f-9ac4-a7ccf55f79ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:26.452050 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:26.463356 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1aeb1dcc-f276-47b1-b48d-7512582d8930 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:26.473413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1aeb1dcc-f276-47b1-b48d-7512582d8930 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:26.475500 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1aeb1dcc-f276-47b1-b48d-7512582d8930 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:26.476872 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1aeb1dcc-f276-47b1-b48d-7512582d8930 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1405 microversion: 2.20 time: 0.186475 Aug 21 01:30:26.477972 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 329/670] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:26 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1405 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:30:26.540026 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-755d6f04-3439-4f9f-9ac4-a7ccf55f79ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:26.549782 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-755d6f04-3439-4f9f-9ac4-a7ccf55f79ca tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1289 microversion: 2.1 time: 0.119327 Aug 21 01:30:26.550576 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 342/671] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:26 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1289 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:26.879263 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-704d64c0-9c6b-400f-9c56-90b0d0f0265e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:26.879944 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-704d64c0-9c6b-400f-9c56-90b0d0f0265e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:26.886316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-704d64c0-9c6b-400f-9c56-90b0d0f0265e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:26.886802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-704d64c0-9c6b-400f-9c56-90b0d0f0265e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:26.887241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-704d64c0-9c6b-400f-9c56-90b0d0f0265e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:26.887938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-704d64c0-9c6b-400f-9c56-90b0d0f0265e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:26.888428 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-704d64c0-9c6b-400f-9c56-90b0d0f0265e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:26.888857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-704d64c0-9c6b-400f-9c56-90b0d0f0265e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:26.958296 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-704d64c0-9c6b-400f-9c56-90b0d0f0265e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:26.967651 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-704d64c0-9c6b-400f-9c56-90b0d0f0265e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1620 microversion: 2.1 time: 0.092574 Aug 21 01:30:26.968706 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 330/672] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:26 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1620 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:26.976899 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-d4c5d034-d75d-4157-b262-e8609c3f408f req-7634877d-1ecb-4946-a5c6-196d7d980090 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eddb6b3f-8ca0-4f71-b371-19c6e65dd095", "name": "network-vif-deleted", "tag": "1e55ecc3-f194-4529-b75c-be5293dba5c9"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:26.987530 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-d4c5d034-d75d-4157-b262-e8609c3f408f req-7634877d-1ecb-4946-a5c6-196d7d980090 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:26.988010 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-d4c5d034-d75d-4157-b262-e8609c3f408f req-7634877d-1ecb-4946-a5c6-196d7d980090 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:26.988690 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-d4c5d034-d75d-4157-b262-e8609c3f408f req-7634877d-1ecb-4946-a5c6-196d7d980090 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:27.000253 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-d4c5d034-d75d-4157-b262-e8609c3f408f req-7634877d-1ecb-4946-a5c6-196d7d980090 service nova] Creating event network-vif-deleted:1e55ecc3-f194-4529-b75c-be5293dba5c9 for instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 on hv-cinder-maste Aug 21 01:30:27.005473 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-d4c5d034-d75d-4157-b262-e8609c3f408f req-7634877d-1ecb-4946-a5c6-196d7d980090 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032548 Aug 21 01:30:27.006222 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 343/673] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:27.501342 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fe32a681-f6ff-4f18-82c5-41d86e1e5506 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:27.502022 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fe32a681-f6ff-4f18-82c5-41d86e1e5506 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:27.510257 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe32a681-f6ff-4f18-82c5-41d86e1e5506 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:27.510735 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe32a681-f6ff-4f18-82c5-41d86e1e5506 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:27.511149 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe32a681-f6ff-4f18-82c5-41d86e1e5506 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:27.511849 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe32a681-f6ff-4f18-82c5-41d86e1e5506 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:27.512329 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe32a681-f6ff-4f18-82c5-41d86e1e5506 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:27.512752 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fe32a681-f6ff-4f18-82c5-41d86e1e5506 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:27.572506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-90222091-ed38-456b-ae9b-c5e70625bdcd tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:27.573842 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-90222091-ed38-456b-ae9b-c5e70625bdcd tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:27.588310 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-90222091-ed38-456b-ae9b-c5e70625bdcd tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:27.590016 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-90222091-ed38-456b-ae9b-c5e70625bdcd tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:27.591121 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-90222091-ed38-456b-ae9b-c5e70625bdcd tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:27.592613 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-90222091-ed38-456b-ae9b-c5e70625bdcd tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:27.593373 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-90222091-ed38-456b-ae9b-c5e70625bdcd tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:27.593972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-90222091-ed38-456b-ae9b-c5e70625bdcd tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:27.678927 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-90222091-ed38-456b-ae9b-c5e70625bdcd tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:27.694662 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fe32a681-f6ff-4f18-82c5-41d86e1e5506 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:27.699343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-90222091-ed38-456b-ae9b-c5e70625bdcd tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1289 microversion: 2.1 time: 0.131519 Aug 21 01:30:27.701242 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 344/674] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:27 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1289 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:27.707082 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fe32a681-f6ff-4f18-82c5-41d86e1e5506 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:27.709112 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fe32a681-f6ff-4f18-82c5-41d86e1e5506 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:27.710439 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fe32a681-f6ff-4f18-82c5-41d86e1e5506 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1405 microversion: 2.20 time: 0.214747 Aug 21 01:30:27.711080 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 331/675] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:27 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1405 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:30:27.991263 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:27.991877 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:28.000413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:28.000413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:28.000413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:28.000882 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:28.001521 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:28.002078 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:28.035072 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:28.035702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:28.041025 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:28.041025 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lazy-loading 'fault' on Instance uuid eddb6b3f-8ca0-4f71-b371-19c6e65dd095 {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:30:28.088644 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:28.098865 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8c2acb2b-39d1-4403-bfd1-e57aa26e0f1b tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1334 microversion: 2.1 time: 0.111675 Aug 21 01:30:28.099482 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 345/676] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:27 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1334 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:28.723013 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-471764aa-d240-475a-bb67-dc30352ac8ba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:28.723834 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-471764aa-d240-475a-bb67-dc30352ac8ba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:28.732285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-0fc9dd75-adc7-4946-a471-ad9078f68ac6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:28.732927 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-0fc9dd75-adc7-4946-a471-ad9078f68ac6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:28.735603 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-471764aa-d240-475a-bb67-dc30352ac8ba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:28.736130 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-471764aa-d240-475a-bb67-dc30352ac8ba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:28.736582 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-471764aa-d240-475a-bb67-dc30352ac8ba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:28.737293 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-471764aa-d240-475a-bb67-dc30352ac8ba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:28.737754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-471764aa-d240-475a-bb67-dc30352ac8ba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:28.738197 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-471764aa-d240-475a-bb67-dc30352ac8ba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:28.739022 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0fc9dd75-adc7-4946-a471-ad9078f68ac6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:28.739513 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0fc9dd75-adc7-4946-a471-ad9078f68ac6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:28.739949 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0fc9dd75-adc7-4946-a471-ad9078f68ac6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:28.740700 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0fc9dd75-adc7-4946-a471-ad9078f68ac6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:28.741186 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0fc9dd75-adc7-4946-a471-ad9078f68ac6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:28.741579 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0fc9dd75-adc7-4946-a471-ad9078f68ac6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:28.828237 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-471764aa-d240-475a-bb67-dc30352ac8ba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:28.836635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-471764aa-d240-475a-bb67-dc30352ac8ba tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1297 microversion: 2.1 time: 0.117626 Aug 21 01:30:28.837210 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 332/677] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:28 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1297 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:28.898765 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-0fc9dd75-adc7-4946-a471-ad9078f68ac6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:28.907575 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-0fc9dd75-adc7-4946-a471-ad9078f68ac6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:28.909144 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-0fc9dd75-adc7-4946-a471-ad9078f68ac6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:28.910349 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-0fc9dd75-adc7-4946-a471-ad9078f68ac6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1596 microversion: 2.20 time: 0.182051 Aug 21 01:30:28.910918 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 346/678] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:30:28 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1596 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:30:28.948549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-30504c2e-a216-470a-aef1-b9a5f496cf17 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:28.949060 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-30504c2e-a216-470a-aef1-b9a5f496cf17 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:28.956071 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30504c2e-a216-470a-aef1-b9a5f496cf17 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:28.956947 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30504c2e-a216-470a-aef1-b9a5f496cf17 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:28.957448 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30504c2e-a216-470a-aef1-b9a5f496cf17 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:28.958221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30504c2e-a216-470a-aef1-b9a5f496cf17 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:28.958743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30504c2e-a216-470a-aef1-b9a5f496cf17 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:28.959204 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30504c2e-a216-470a-aef1-b9a5f496cf17 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:29.044758 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-30504c2e-a216-470a-aef1-b9a5f496cf17 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.101571 Aug 21 01:30:29.045430 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 333/679] 10.222.0.47 () {60 vars in 1316 bytes} [Sun Aug 21 01:30:28 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/os-interface => generated 284 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:30:29.120964 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:29.121614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:29.136709 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:29.137682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:29.138323 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:29.139238 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:29.139861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:29.140476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:29.172805 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:29.173489 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:29.173937 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:29.174826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lazy-loading 'fault' on Instance uuid eddb6b3f-8ca0-4f71-b371-19c6e65dd095 {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:30:29.242950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:29.251725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-464fc791-a0c0-4079-887c-961efe3bb3bd tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 200 len: 1334 microversion: 2.1 time: 0.135026 Aug 21 01:30:29.252519 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 347/680] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:29 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 1334 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:29.866308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6066eca7-1329-4ab3-b9c5-baf8534ac634 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:29.868012 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6066eca7-1329-4ab3-b9c5-baf8534ac634 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:29.880198 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6066eca7-1329-4ab3-b9c5-baf8534ac634 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:29.880198 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6066eca7-1329-4ab3-b9c5-baf8534ac634 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:29.880198 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6066eca7-1329-4ab3-b9c5-baf8534ac634 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:29.880198 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6066eca7-1329-4ab3-b9c5-baf8534ac634 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:29.880569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6066eca7-1329-4ab3-b9c5-baf8534ac634 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:29.880569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6066eca7-1329-4ab3-b9c5-baf8534ac634 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:30.135728 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6066eca7-1329-4ab3-b9c5-baf8534ac634 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:30.144948 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6066eca7-1329-4ab3-b9c5-baf8534ac634 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.289693 Aug 21 01:30:30.146788 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 334/681] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:29 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:30.273615 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-dd79f0aa-6146-41a0-8486-c08af54d17c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:30.277668 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-dd79f0aa-6146-41a0-8486-c08af54d17c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: eddb6b3f-8ca0-4f71-b371-19c6e65dd095] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:30.291056 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dd79f0aa-6146-41a0-8486-c08af54d17c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:30.291550 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dd79f0aa-6146-41a0-8486-c08af54d17c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:30.292227 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dd79f0aa-6146-41a0-8486-c08af54d17c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:30.292939 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dd79f0aa-6146-41a0-8486-c08af54d17c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:30.293391 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dd79f0aa-6146-41a0-8486-c08af54d17c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:30.293814 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dd79f0aa-6146-41a0-8486-c08af54d17c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:30.312903 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-dd79f0aa-6146-41a0-8486-c08af54d17c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] HTTP exception thrown: Instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 could not be found. Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-dd79f0aa-6146-41a0-8486-c08af54d17c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Request method failure captured: Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 HTTP/1.1 Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYl2grkk6r6DOR4WcdCDQaZ1LKSwt5ty8EvPKEsirHl25Bbz12fukVsuTZFRPo5OnAmJDHzJfldGu7OE-fBThLL0sUCHPYzsZbzZPojBGa0s89nUPxFfxXEogZCrYm2ZzOthYXvM-jM7ah9S1dz6CN84xJGSagaAUIMSBiUILa4u8mo Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 9774c354b8b048768f2071acfb48a4bf Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:30:30.314682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:30:30.316414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:30:30.316414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:30:30.316414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 9774c354b8b048768f2071acfb48a4bf Aug 21 01:30:30.316414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:30:30.316414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-ServerRescueNegativeTestJSON-1204886246-project Aug 21 01:30:30.316414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:30:30.316414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:30:30.316414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: d84b6d05decb47c7b55ac782d5839c1a Aug 21 01:30:30.316414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1204886246-project Aug 21 01:30:30.316414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:30:30.316414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 could not be found. Aug 21 01:30:30.316414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': 'eddb6b3f-8ca0-4f71-b371-19c6e65dd095'} Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 could not be found. Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:30:30.317294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 could not be found. Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 could not be found. Aug 21 01:30:30.318729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:30:30.320863 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-dd79f0aa-6146-41a0-8486-c08af54d17c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Returning 404 to user: Instance eddb6b3f-8ca0-4f71-b371-19c6e65dd095 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:30:30.322778 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-dd79f0aa-6146-41a0-8486-c08af54d17c0 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095" status: 404 len: 111 microversion: 2.1 time: 0.052419 Aug 21 01:30:30.323573 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 348/682] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:30 2022] GET /compute/v2.1/servers/eddb6b3f-8ca0-4f71-b371-19c6e65dd095 => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:30:30.335723 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8a900183-711e-42f4-822f-18224d2ddf9e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '.version_select at 0x7f9da33f8af0>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:30.359143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-ce7c6f29-2148-47a7-9307-7964139998f9 req-21c144df-fd11-4bce-ac4e-28977e6c8cd8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a5b475aa-486a-4957-8b54-aa1ee7438a8a", "tag": "d3beeb80-5980-4de1-b832-2d4a1e1dae1c"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:30.369882 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ce7c6f29-2148-47a7-9307-7964139998f9 req-21c144df-fd11-4bce-ac4e-28977e6c8cd8 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:30.370340 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ce7c6f29-2148-47a7-9307-7964139998f9 req-21c144df-fd11-4bce-ac4e-28977e6c8cd8 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:30.370804 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ce7c6f29-2148-47a7-9307-7964139998f9 req-21c144df-fd11-4bce-ac4e-28977e6c8cd8 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:30.383245 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-ce7c6f29-2148-47a7-9307-7964139998f9 req-21c144df-fd11-4bce-ac4e-28977e6c8cd8 service nova] Creating event network-changed:d3beeb80-5980-4de1-b832-2d4a1e1dae1c for instance a5b475aa-486a-4957-8b54-aa1ee7438a8a on hv-cinder-maste Aug 21 01:30:30.400652 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-ce7c6f29-2148-47a7-9307-7964139998f9 req-21c144df-fd11-4bce-ac4e-28977e6c8cd8 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046631 Aug 21 01:30:30.402772 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 349/683] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:30:30.429130 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8a900183-711e-42f4-822f-18224d2ddf9e tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-980024713" status: 202 len: 0 microversion: 2.1 time: 0.101224 Aug 21 01:30:30.429746 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 335/684] 10.222.0.47 () {58 vars in 1218 bytes} [Sun Aug 21 01:30:30 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-980024713 => generated 0 bytes in 103 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:30:31.112508 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-201e7d41-abbe-42cc-bed3-a27b155d068f req-31a06744-fc29-4f26-bf93-3b6b80933594 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9a075e00-151c-4ac4-b454-deea3de38ef7", "tag": "306889a6-f7da-46ac-91cc-f0f0db7e57c3"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:31.134246 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-201e7d41-abbe-42cc-bed3-a27b155d068f req-31a06744-fc29-4f26-bf93-3b6b80933594 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:31.137273 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-201e7d41-abbe-42cc-bed3-a27b155d068f req-31a06744-fc29-4f26-bf93-3b6b80933594 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:31.138259 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-201e7d41-abbe-42cc-bed3-a27b155d068f req-31a06744-fc29-4f26-bf93-3b6b80933594 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:31.171180 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-baa9d4c5-7427-4778-b3ac-f98d0e317828 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:31.171805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-baa9d4c5-7427-4778-b3ac-f98d0e317828 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:31.181877 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-201e7d41-abbe-42cc-bed3-a27b155d068f req-31a06744-fc29-4f26-bf93-3b6b80933594 service nova] Creating event network-changed:306889a6-f7da-46ac-91cc-f0f0db7e57c3 for instance 9a075e00-151c-4ac4-b454-deea3de38ef7 on hv-cinder-maste Aug 21 01:30:31.198388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-baa9d4c5-7427-4778-b3ac-f98d0e317828 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:31.198388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-baa9d4c5-7427-4778-b3ac-f98d0e317828 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:31.198388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-baa9d4c5-7427-4778-b3ac-f98d0e317828 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:31.198388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-baa9d4c5-7427-4778-b3ac-f98d0e317828 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:31.200321 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-baa9d4c5-7427-4778-b3ac-f98d0e317828 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:31.200321 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-baa9d4c5-7427-4778-b3ac-f98d0e317828 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:31.200629 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-201e7d41-abbe-42cc-bed3-a27b155d068f req-31a06744-fc29-4f26-bf93-3b6b80933594 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.090482 Aug 21 01:30:31.200629 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 350/685] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:31.508837 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-baa9d4c5-7427-4778-b3ac-f98d0e317828 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:31.517580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-baa9d4c5-7427-4778-b3ac-f98d0e317828 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.354094 Aug 21 01:30:31.518150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 336/686] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:31 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:32.536597 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:32.537416 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=14 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:32.541624 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3d6aecb9-8c48-488c-9f78-a4b7bdcf6e25 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:32.542404 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3d6aecb9-8c48-488c-9f78-a4b7bdcf6e25 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:32.548634 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d6aecb9-8c48-488c-9f78-a4b7bdcf6e25 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:32.549144 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d6aecb9-8c48-488c-9f78-a4b7bdcf6e25 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:32.549566 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d6aecb9-8c48-488c-9f78-a4b7bdcf6e25 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:32.550287 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d6aecb9-8c48-488c-9f78-a4b7bdcf6e25 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:32.550743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d6aecb9-8c48-488c-9f78-a4b7bdcf6e25 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:32.551182 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d6aecb9-8c48-488c-9f78-a4b7bdcf6e25 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:32.694368 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3d6aecb9-8c48-488c-9f78-a4b7bdcf6e25 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:32.703060 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3d6aecb9-8c48-488c-9f78-a4b7bdcf6e25 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.167972 Aug 21 01:30:32.703619 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 351/687] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:32 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:33.724491 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-720f4754-ee9f-4731-bd9d-fba17b48b5e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:33.725193 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-720f4754-ee9f-4731-bd9d-fba17b48b5e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:33.733111 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-720f4754-ee9f-4731-bd9d-fba17b48b5e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:33.733703 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-720f4754-ee9f-4731-bd9d-fba17b48b5e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:33.734224 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-720f4754-ee9f-4731-bd9d-fba17b48b5e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:33.734957 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-720f4754-ee9f-4731-bd9d-fba17b48b5e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:33.735417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-720f4754-ee9f-4731-bd9d-fba17b48b5e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:33.736055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-720f4754-ee9f-4731-bd9d-fba17b48b5e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:33.874786 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-720f4754-ee9f-4731-bd9d-fba17b48b5e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:33.882732 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-720f4754-ee9f-4731-bd9d-fba17b48b5e4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.162368 Aug 21 01:30:33.883307 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 337/688] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:33 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:34.903792 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9fa190c7-1e6e-45b6-8c07-d67123b75739 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:34.904433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9fa190c7-1e6e-45b6-8c07-d67123b75739 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:34.911720 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9fa190c7-1e6e-45b6-8c07-d67123b75739 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:34.912244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9fa190c7-1e6e-45b6-8c07-d67123b75739 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:34.912689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9fa190c7-1e6e-45b6-8c07-d67123b75739 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:34.913424 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9fa190c7-1e6e-45b6-8c07-d67123b75739 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:34.914005 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9fa190c7-1e6e-45b6-8c07-d67123b75739 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:34.914434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9fa190c7-1e6e-45b6-8c07-d67123b75739 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:35.126658 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9fa190c7-1e6e-45b6-8c07-d67123b75739 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:35.138018 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9fa190c7-1e6e-45b6-8c07-d67123b75739 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.238922 Aug 21 01:30:35.139079 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 352/689] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:34 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:36.159659 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1e1ae1ae-659f-4e7b-b86b-ad391769e911 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:36.160260 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1e1ae1ae-659f-4e7b-b86b-ad391769e911 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:36.166215 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1e1ae1ae-659f-4e7b-b86b-ad391769e911 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:36.166690 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1e1ae1ae-659f-4e7b-b86b-ad391769e911 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:36.167175 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1e1ae1ae-659f-4e7b-b86b-ad391769e911 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:36.167808 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1e1ae1ae-659f-4e7b-b86b-ad391769e911 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:36.168287 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1e1ae1ae-659f-4e7b-b86b-ad391769e911 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:36.168756 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1e1ae1ae-659f-4e7b-b86b-ad391769e911 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:36.322933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1e1ae1ae-659f-4e7b-b86b-ad391769e911 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:36.331482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1e1ae1ae-659f-4e7b-b86b-ad391769e911 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.176069 Aug 21 01:30:36.332062 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 338/690] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:36 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:37.353072 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-80709351-4043-41be-b9e6-b88887c70636 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:37.353708 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-80709351-4043-41be-b9e6-b88887c70636 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:37.360286 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-80709351-4043-41be-b9e6-b88887c70636 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:37.364670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-80709351-4043-41be-b9e6-b88887c70636 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:37.364670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-80709351-4043-41be-b9e6-b88887c70636 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:37.364670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-80709351-4043-41be-b9e6-b88887c70636 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:37.364670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-80709351-4043-41be-b9e6-b88887c70636 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:37.365071 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-80709351-4043-41be-b9e6-b88887c70636 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:37.497845 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-80709351-4043-41be-b9e6-b88887c70636 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:37.506408 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-80709351-4043-41be-b9e6-b88887c70636 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.157398 Aug 21 01:30:37.507187 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 353/691] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:37 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:38.530790 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a4fe1dbf-e67a-43e3-8c7c-89d4a325a587 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:38.531396 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a4fe1dbf-e67a-43e3-8c7c-89d4a325a587 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:38.538057 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4fe1dbf-e67a-43e3-8c7c-89d4a325a587 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:38.538536 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4fe1dbf-e67a-43e3-8c7c-89d4a325a587 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:38.538973 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4fe1dbf-e67a-43e3-8c7c-89d4a325a587 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:38.539669 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4fe1dbf-e67a-43e3-8c7c-89d4a325a587 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:38.540142 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4fe1dbf-e67a-43e3-8c7c-89d4a325a587 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:38.540569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4fe1dbf-e67a-43e3-8c7c-89d4a325a587 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:38.545261 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:38.676740 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a4fe1dbf-e67a-43e3-8c7c-89d4a325a587 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:38.686806 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a4fe1dbf-e67a-43e3-8c7c-89d4a325a587 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.159876 Aug 21 01:30:38.687387 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 339/692] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:38 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:39.709502 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9ee9e450-1df0-437d-be0d-62e414ea2267 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:39.710158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9ee9e450-1df0-437d-be0d-62e414ea2267 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:39.716713 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ee9e450-1df0-437d-be0d-62e414ea2267 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:39.717156 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ee9e450-1df0-437d-be0d-62e414ea2267 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:39.717601 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ee9e450-1df0-437d-be0d-62e414ea2267 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:39.718314 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ee9e450-1df0-437d-be0d-62e414ea2267 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:39.718823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ee9e450-1df0-437d-be0d-62e414ea2267 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:39.719431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9ee9e450-1df0-437d-be0d-62e414ea2267 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:39.911174 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9ee9e450-1df0-437d-be0d-62e414ea2267 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:39.920474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9ee9e450-1df0-437d-be0d-62e414ea2267 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.216114 Aug 21 01:30:39.921047 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 354/693] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:39 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:40.879041 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4b6204aa-5e95-43b1-9046-d1cffefd50a3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:40.879550 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4b6204aa-5e95-43b1-9046-d1cffefd50a3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:40.887620 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4b6204aa-5e95-43b1-9046-d1cffefd50a3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:40.888191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4b6204aa-5e95-43b1-9046-d1cffefd50a3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:40.888596 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4b6204aa-5e95-43b1-9046-d1cffefd50a3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:40.889308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4b6204aa-5e95-43b1-9046-d1cffefd50a3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:40.889787 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4b6204aa-5e95-43b1-9046-d1cffefd50a3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:40.890219 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4b6204aa-5e95-43b1-9046-d1cffefd50a3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:40.917496 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4b6204aa-5e95-43b1-9046-d1cffefd50a3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:30:40.945486 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b44a55e1-8708-433a-bdbc-8eb50da5af31 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:40.947202 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b44a55e1-8708-433a-bdbc-8eb50da5af31 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:40.955595 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b44a55e1-8708-433a-bdbc-8eb50da5af31 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:40.956089 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b44a55e1-8708-433a-bdbc-8eb50da5af31 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:40.956549 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b44a55e1-8708-433a-bdbc-8eb50da5af31 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:40.957252 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b44a55e1-8708-433a-bdbc-8eb50da5af31 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:40.957853 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b44a55e1-8708-433a-bdbc-8eb50da5af31 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:40.958277 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b44a55e1-8708-433a-bdbc-8eb50da5af31 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:41.111218 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b44a55e1-8708-433a-bdbc-8eb50da5af31 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:41.122039 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b44a55e1-8708-433a-bdbc-8eb50da5af31 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.180543 Aug 21 01:30:41.122888 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 355/694] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:40 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:41.334219 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4b6204aa-5e95-43b1-9046-d1cffefd50a3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "DELETE /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 204 len: 0 microversion: 2.1 time: 0.458890 Aug 21 01:30:41.337252 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 340/695] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:30:40 2022] DELETE /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 0 bytes in 460 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:30:41.343418 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9dad8d39-e4ab-4456-8a9d-b50d8b3fd936 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:41.344070 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9dad8d39-e4ab-4456-8a9d-b50d8b3fd936 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:41.350547 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9dad8d39-e4ab-4456-8a9d-b50d8b3fd936 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:41.351051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9dad8d39-e4ab-4456-8a9d-b50d8b3fd936 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:41.351494 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9dad8d39-e4ab-4456-8a9d-b50d8b3fd936 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:41.352686 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9dad8d39-e4ab-4456-8a9d-b50d8b3fd936 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:41.352809 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9dad8d39-e4ab-4456-8a9d-b50d8b3fd936 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:41.353142 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9dad8d39-e4ab-4456-8a9d-b50d8b3fd936 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:41.522317 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9dad8d39-e4ab-4456-8a9d-b50d8b3fd936 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:41.533385 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9dad8d39-e4ab-4456-8a9d-b50d8b3fd936 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1507 microversion: 2.1 time: 0.193994 Aug 21 01:30:41.534054 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 356/696] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:41 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1507 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:41.965997 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-d88bcaa7-3bed-465e-ac9a-2ca4c95fa643 req-56da67d7-ae49-42ac-8229-a0b7422d76a4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a075e00-151c-4ac4-b454-deea3de38ef7", "name": "network-vif-plugged", "status": "completed", "tag": "306889a6-f7da-46ac-91cc-f0f0db7e57c3"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:41.977692 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-d88bcaa7-3bed-465e-ac9a-2ca4c95fa643 req-56da67d7-ae49-42ac-8229-a0b7422d76a4 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:41.978175 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-d88bcaa7-3bed-465e-ac9a-2ca4c95fa643 req-56da67d7-ae49-42ac-8229-a0b7422d76a4 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:41.978618 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-d88bcaa7-3bed-465e-ac9a-2ca4c95fa643 req-56da67d7-ae49-42ac-8229-a0b7422d76a4 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:41.994435 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-d88bcaa7-3bed-465e-ac9a-2ca4c95fa643 req-56da67d7-ae49-42ac-8229-a0b7422d76a4 service nova] Creating event network-vif-plugged:306889a6-f7da-46ac-91cc-f0f0db7e57c3 for instance 9a075e00-151c-4ac4-b454-deea3de38ef7 on hv-cinder-maste Aug 21 01:30:42.001168 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-d88bcaa7-3bed-465e-ac9a-2ca4c95fa643 req-56da67d7-ae49-42ac-8229-a0b7422d76a4 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040735 Aug 21 01:30:42.002540 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 341/697] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:42.147217 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-42a74ba4-57cd-4d1d-bdf7-e5bb44fb0cc4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:42.147828 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-42a74ba4-57cd-4d1d-bdf7-e5bb44fb0cc4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:42.154348 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-42a74ba4-57cd-4d1d-bdf7-e5bb44fb0cc4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:42.154833 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-42a74ba4-57cd-4d1d-bdf7-e5bb44fb0cc4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:42.155264 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-42a74ba4-57cd-4d1d-bdf7-e5bb44fb0cc4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:42.155956 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-42a74ba4-57cd-4d1d-bdf7-e5bb44fb0cc4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:42.156462 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-42a74ba4-57cd-4d1d-bdf7-e5bb44fb0cc4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:42.156933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-42a74ba4-57cd-4d1d-bdf7-e5bb44fb0cc4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:42.322778 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-42a74ba4-57cd-4d1d-bdf7-e5bb44fb0cc4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:42.331407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-42a74ba4-57cd-4d1d-bdf7-e5bb44fb0cc4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.188102 Aug 21 01:30:42.332081 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 357/698] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:42 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:42.555816 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-be1f3cc9-2eb6-4f27-aaf9-baa4b02936a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:42.556595 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-be1f3cc9-2eb6-4f27-aaf9-baa4b02936a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:42.564239 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-be1f3cc9-2eb6-4f27-aaf9-baa4b02936a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:42.564239 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-be1f3cc9-2eb6-4f27-aaf9-baa4b02936a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:42.564624 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-be1f3cc9-2eb6-4f27-aaf9-baa4b02936a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:42.565279 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-be1f3cc9-2eb6-4f27-aaf9-baa4b02936a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:42.565737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-be1f3cc9-2eb6-4f27-aaf9-baa4b02936a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:42.566160 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-be1f3cc9-2eb6-4f27-aaf9-baa4b02936a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:42.726730 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-be1f3cc9-2eb6-4f27-aaf9-baa4b02936a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:42.739109 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-be1f3cc9-2eb6-4f27-aaf9-baa4b02936a4 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1507 microversion: 2.1 time: 0.187535 Aug 21 01:30:42.740024 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 342/699] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:42 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1507 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:43.355938 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ede731a6-0b22-4bbf-8786-656befc45b09 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:43.356765 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-ede731a6-0b22-4bbf-8786-656befc45b09 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:43.364090 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ede731a6-0b22-4bbf-8786-656befc45b09 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:43.364555 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ede731a6-0b22-4bbf-8786-656befc45b09 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:43.364988 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ede731a6-0b22-4bbf-8786-656befc45b09 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:43.365690 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ede731a6-0b22-4bbf-8786-656befc45b09 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:43.366180 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ede731a6-0b22-4bbf-8786-656befc45b09 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:43.366619 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ede731a6-0b22-4bbf-8786-656befc45b09 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:43.522083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ede731a6-0b22-4bbf-8786-656befc45b09 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:43.531049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ede731a6-0b22-4bbf-8786-656befc45b09 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.181363 Aug 21 01:30:43.531702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 358/700] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:43 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:43.734160 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4015bcb4-b0b4-44d1-bc10-75cf6b93c471 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:43.735432 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4015bcb4-b0b4-44d1-bc10-75cf6b93c471 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:43.746026 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4015bcb4-b0b4-44d1-bc10-75cf6b93c471 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:43.749709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4015bcb4-b0b4-44d1-bc10-75cf6b93c471 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:43.749709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4015bcb4-b0b4-44d1-bc10-75cf6b93c471 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:43.749709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4015bcb4-b0b4-44d1-bc10-75cf6b93c471 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:43.749709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4015bcb4-b0b4-44d1-bc10-75cf6b93c471 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:43.750280 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4015bcb4-b0b4-44d1-bc10-75cf6b93c471 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:43.760419 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-89f711da-c90f-4c13-8439-8df6fad4392a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:43.761127 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-89f711da-c90f-4c13-8439-8df6fad4392a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:43.767853 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-89f711da-c90f-4c13-8439-8df6fad4392a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:43.768431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-89f711da-c90f-4c13-8439-8df6fad4392a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:43.768903 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-89f711da-c90f-4c13-8439-8df6fad4392a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:43.770092 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-89f711da-c90f-4c13-8439-8df6fad4392a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:43.770092 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-89f711da-c90f-4c13-8439-8df6fad4392a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:43.770526 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-89f711da-c90f-4c13-8439-8df6fad4392a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:43.870549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-4015bcb4-b0b4-44d1-bc10-75cf6b93c471 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lazy-loading 'services' on Instance uuid bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:30:43.925933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-89f711da-c90f-4c13-8439-8df6fad4392a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:43.936165 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-89f711da-c90f-4c13-8439-8df6fad4392a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1507 microversion: 2.1 time: 0.179492 Aug 21 01:30:43.936719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 359/701] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:43 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1507 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:44.111848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4015bcb4-b0b4-44d1-bc10-75cf6b93c471 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "DELETE /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad/os-volume_attachments/fdcfe337-2074-4bbd-bad1-862c9ae74f17" status: 202 len: 0 microversion: 2.1 time: 0.382179 Aug 21 01:30:44.112506 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 343/702] 10.222.0.47 () {58 vars in 1416 bytes} [Sun Aug 21 01:30:43 2022] DELETE /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad/os-volume_attachments/fdcfe337-2074-4bbd-bad1-862c9ae74f17 => generated 0 bytes in 384 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 21 01:30:44.555500 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f4aa6066-92c7-4b2c-a978-97eee7acd22e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:44.556026 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f4aa6066-92c7-4b2c-a978-97eee7acd22e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:44.562967 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f4aa6066-92c7-4b2c-a978-97eee7acd22e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:44.563462 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f4aa6066-92c7-4b2c-a978-97eee7acd22e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:44.563901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f4aa6066-92c7-4b2c-a978-97eee7acd22e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:44.564940 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f4aa6066-92c7-4b2c-a978-97eee7acd22e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:44.565398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f4aa6066-92c7-4b2c-a978-97eee7acd22e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:44.565821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f4aa6066-92c7-4b2c-a978-97eee7acd22e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:44.865806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f4aa6066-92c7-4b2c-a978-97eee7acd22e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:44.870070 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=100 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:44.875461 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f4aa6066-92c7-4b2c-a978-97eee7acd22e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.324570 Aug 21 01:30:44.882105 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 360/703] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:44 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:44.958393 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d53bac53-89a8-4c6e-a489-645825bdc21c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:44.959088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d53bac53-89a8-4c6e-a489-645825bdc21c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:44.966650 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d53bac53-89a8-4c6e-a489-645825bdc21c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:44.967140 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d53bac53-89a8-4c6e-a489-645825bdc21c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:44.967581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d53bac53-89a8-4c6e-a489-645825bdc21c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:44.969174 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d53bac53-89a8-4c6e-a489-645825bdc21c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:44.969642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d53bac53-89a8-4c6e-a489-645825bdc21c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:44.970096 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d53bac53-89a8-4c6e-a489-645825bdc21c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:45.134726 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d53bac53-89a8-4c6e-a489-645825bdc21c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:45.148763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d53bac53-89a8-4c6e-a489-645825bdc21c tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1465 microversion: 2.1 time: 0.195231 Aug 21 01:30:45.149943 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 344/704] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:44 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1465 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:45.227432 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-b2df5c2b-4ed4-4412-a0b6-2e33fce30d05 req-2618599e-bf89-41a3-9a16-06b286860cb3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "80c72a3e-79b5-48fa-9c5f-26a858b589de", "name": "network-vif-deleted", "tag": "f7aa7447-c670-4701-a6ac-18f99415ce92"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:45.243587 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-b2df5c2b-4ed4-4412-a0b6-2e33fce30d05 req-2618599e-bf89-41a3-9a16-06b286860cb3 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:45.244230 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-b2df5c2b-4ed4-4412-a0b6-2e33fce30d05 req-2618599e-bf89-41a3-9a16-06b286860cb3 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:45.244976 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-b2df5c2b-4ed4-4412-a0b6-2e33fce30d05 req-2618599e-bf89-41a3-9a16-06b286860cb3 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:45.264708 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-b2df5c2b-4ed4-4412-a0b6-2e33fce30d05 req-2618599e-bf89-41a3-9a16-06b286860cb3 service nova] Creating event network-vif-deleted:f7aa7447-c670-4701-a6ac-18f99415ce92 for instance 80c72a3e-79b5-48fa-9c5f-26a858b589de on hv-cinder-maste Aug 21 01:30:45.271322 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-b2df5c2b-4ed4-4412-a0b6-2e33fce30d05 req-2618599e-bf89-41a3-9a16-06b286860cb3 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.141129 Aug 21 01:30:45.271933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 361/705] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:45.898115 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-08331f85-519b-4c01-8841-8cd72690692b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:45.898735 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-08331f85-519b-4c01-8841-8cd72690692b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:45.907656 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-08331f85-519b-4c01-8841-8cd72690692b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:45.908818 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-08331f85-519b-4c01-8841-8cd72690692b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:45.909578 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-08331f85-519b-4c01-8841-8cd72690692b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:45.910753 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-08331f85-519b-4c01-8841-8cd72690692b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:45.912069 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-08331f85-519b-4c01-8841-8cd72690692b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:45.912506 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-08331f85-519b-4c01-8841-8cd72690692b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:46.077753 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-08331f85-519b-4c01-8841-8cd72690692b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:46.087979 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-08331f85-519b-4c01-8841-8cd72690692b tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1360 microversion: 2.1 time: 0.194938 Aug 21 01:30:46.088593 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 345/706] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:45 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1360 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:46.174798 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:46.175439 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:46.184305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:46.184748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:46.185229 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:46.185826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:46.186256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:46.187133 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:46.234267 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:46.234831 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:46.235277 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:46.235729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lazy-loading 'fault' on Instance uuid 80c72a3e-79b5-48fa-9c5f-26a858b589de {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:30:46.283253 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:46.294281 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9badd025-3b6f-4b88-85fa-6370136f0028 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1312 microversion: 2.1 time: 0.124323 Aug 21 01:30:46.294970 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 362/707] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:46 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1312 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:47.118540 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b9b9f63a-66c2-4ce3-809e-5c17603a32b9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:47.118540 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-b9b9f63a-66c2-4ce3-809e-5c17603a32b9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:47.118540 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b9b9f63a-66c2-4ce3-809e-5c17603a32b9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:47.118540 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b9b9f63a-66c2-4ce3-809e-5c17603a32b9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:47.119669 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b9b9f63a-66c2-4ce3-809e-5c17603a32b9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:47.119669 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b9b9f63a-66c2-4ce3-809e-5c17603a32b9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:47.119669 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b9b9f63a-66c2-4ce3-809e-5c17603a32b9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:47.119669 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b9b9f63a-66c2-4ce3-809e-5c17603a32b9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:47.289659 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b9b9f63a-66c2-4ce3-809e-5c17603a32b9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:47.298493 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b9b9f63a-66c2-4ce3-809e-5c17603a32b9 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1546 microversion: 2.1 time: 0.194095 Aug 21 01:30:47.299073 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 346/708] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:47 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1546 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:47.321469 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:47.322914 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:47.332199 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c510d2ef-69e4-4bef-b4ad-755a7bc45ee4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:47.332199 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c510d2ef-69e4-4bef-b4ad-755a7bc45ee4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:47.336394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c510d2ef-69e4-4bef-b4ad-755a7bc45ee4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:47.344615 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c510d2ef-69e4-4bef-b4ad-755a7bc45ee4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:47.344615 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c510d2ef-69e4-4bef-b4ad-755a7bc45ee4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:47.344615 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c510d2ef-69e4-4bef-b4ad-755a7bc45ee4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:47.344615 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c510d2ef-69e4-4bef-b4ad-755a7bc45ee4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:47.345116 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:47.345236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c510d2ef-69e4-4bef-b4ad-755a7bc45ee4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:47.345344 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:47.345344 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:47.345344 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:47.345344 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:47.345848 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:47.430400 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:47.432698 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:47.435347 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:47.436399 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lazy-loading 'fault' on Instance uuid 80c72a3e-79b5-48fa-9c5f-26a858b589de {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:30:47.445964 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c510d2ef-69e4-4bef-b4ad-755a7bc45ee4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.123398 Aug 21 01:30:47.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 347/709] 10.222.0.47 () {58 vars in 1275 bytes} [Sun Aug 21 01:30:47 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-interface => generated 284 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:30:47.473226 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-08d69a4b-471d-43e0-a9a8-494a64101641 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:47.474431 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-08d69a4b-471d-43e0-a9a8-494a64101641 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:47.484839 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-08d69a4b-471d-43e0-a9a8-494a64101641 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:47.492611 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-08d69a4b-471d-43e0-a9a8-494a64101641 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:47.492611 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-08d69a4b-471d-43e0-a9a8-494a64101641 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:47.492611 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-08d69a4b-471d-43e0-a9a8-494a64101641 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:47.492611 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-08d69a4b-471d-43e0-a9a8-494a64101641 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:47.493302 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-08d69a4b-471d-43e0-a9a8-494a64101641 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:47.525074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-08d69a4b-471d-43e0-a9a8-494a64101641 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:30:47.534811 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:47.555785 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-56e8fafa-c172-42ca-b98a-535e13768d51 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 200 len: 1312 microversion: 2.1 time: 0.239702 Aug 21 01:30:47.556519 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 363/710] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:47 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 1312 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:47.849583 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-08d69a4b-471d-43e0-a9a8-494a64101641 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "DELETE /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 204 len: 0 microversion: 2.1 time: 0.385461 Aug 21 01:30:47.850932 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 348/711] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:30:47 2022] DELETE /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 0 bytes in 389 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:30:47.860566 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-94fc969c-bb26-404f-91cb-db2b0f09154d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:47.863046 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-94fc969c-bb26-404f-91cb-db2b0f09154d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:47.870773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-94fc969c-bb26-404f-91cb-db2b0f09154d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:47.875129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-94fc969c-bb26-404f-91cb-db2b0f09154d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:47.875129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-94fc969c-bb26-404f-91cb-db2b0f09154d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:47.875129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-94fc969c-bb26-404f-91cb-db2b0f09154d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:47.875129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-94fc969c-bb26-404f-91cb-db2b0f09154d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:47.875525 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-94fc969c-bb26-404f-91cb-db2b0f09154d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:48.161292 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-94fc969c-bb26-404f-91cb-db2b0f09154d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:48.171123 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-94fc969c-bb26-404f-91cb-db2b0f09154d tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1514 microversion: 2.1 time: 0.314673 Aug 21 01:30:48.176297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 364/712] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:47 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1514 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:48.581110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-74783249-ca53-46a5-ba99-30d0f79e8993 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:48.582199 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-74783249-ca53-46a5-ba99-30d0f79e8993 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 80c72a3e-79b5-48fa-9c5f-26a858b589de] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:48.590082 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-74783249-ca53-46a5-ba99-30d0f79e8993 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:48.590821 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-74783249-ca53-46a5-ba99-30d0f79e8993 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:48.591348 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-74783249-ca53-46a5-ba99-30d0f79e8993 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:48.593016 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-74783249-ca53-46a5-ba99-30d0f79e8993 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:48.593483 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-74783249-ca53-46a5-ba99-30d0f79e8993 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:48.595376 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-74783249-ca53-46a5-ba99-30d0f79e8993 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:48.616632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-74783249-ca53-46a5-ba99-30d0f79e8993 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] HTTP exception thrown: Instance 80c72a3e-79b5-48fa-9c5f-26a858b589de could not be found. Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-74783249-ca53-46a5-ba99-30d0f79e8993 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Request method failure captured: Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de HTTP/1.1 Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYl2grkk6r6DOR4WcdCDQaZ1LKSwt5ty8EvPKEsirHl25Bbz12fukVsuTZFRPo5OnAmJDHzJfldGu7OE-fBThLL0sUCHPYzsZbzZPojBGa0s89nUPxFfxXEogZCrYm2ZzOthYXvM-jM7ah9S1dz6CN84xJGSagaAUIMSBiUILa4u8mo Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 9774c354b8b048768f2071acfb48a4bf Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:30:48.621911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:30:48.623365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:30:48.623365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:30:48.623365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 9774c354b8b048768f2071acfb48a4bf Aug 21 01:30:48.623365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:30:48.623365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-ServerRescueNegativeTestJSON-1204886246-project Aug 21 01:30:48.623365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:30:48.623365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:30:48.623365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: d84b6d05decb47c7b55ac782d5839c1a Aug 21 01:30:48.623365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1204886246-project Aug 21 01:30:48.623365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:30:48.623365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 80c72a3e-79b5-48fa-9c5f-26a858b589de could not be found. Aug 21 01:30:48.623365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '80c72a3e-79b5-48fa-9c5f-26a858b589de'} Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 80c72a3e-79b5-48fa-9c5f-26a858b589de could not be found. Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:30:48.624181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 80c72a3e-79b5-48fa-9c5f-26a858b589de could not be found. Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 80c72a3e-79b5-48fa-9c5f-26a858b589de could not be found. Aug 21 01:30:48.625413 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:48.626490 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-74783249-ca53-46a5-ba99-30d0f79e8993 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Returning 404 to user: Instance 80c72a3e-79b5-48fa-9c5f-26a858b589de could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:30:48.626490 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-74783249-ca53-46a5-ba99-30d0f79e8993 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de" status: 404 len: 111 microversion: 2.1 time: 0.049504 Aug 21 01:30:48.626490 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 349/713] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:48 2022] GET /compute/v2.1/servers/80c72a3e-79b5-48fa-9c5f-26a858b589de => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:30:48.651245 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-208b6223-7525-41e5-bc2c-67d40640e08f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:48.652541 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-208b6223-7525-41e5-bc2c-67d40640e08f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:48.656060 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=4 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:48.665257 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-208b6223-7525-41e5-bc2c-67d40640e08f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:48.665887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-208b6223-7525-41e5-bc2c-67d40640e08f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:48.666329 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-208b6223-7525-41e5-bc2c-67d40640e08f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:48.667041 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-208b6223-7525-41e5-bc2c-67d40640e08f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:48.667714 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-208b6223-7525-41e5-bc2c-67d40640e08f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:48.668380 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-208b6223-7525-41e5-bc2c-67d40640e08f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:48.701313 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-208b6223-7525-41e5-bc2c-67d40640e08f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Going to try to terminate instance {{(pid=121277) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:30:48.750946 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-83152ffd-dcc4-4ff3-bafd-3a2118997865 req-696006cc-346f-43f6-8099-b5c7710615fd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9a075e00-151c-4ac4-b454-deea3de38ef7", "tag": "306889a6-f7da-46ac-91cc-f0f0db7e57c3"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:48.761671 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-83152ffd-dcc4-4ff3-bafd-3a2118997865 req-696006cc-346f-43f6-8099-b5c7710615fd service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:48.762272 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-83152ffd-dcc4-4ff3-bafd-3a2118997865 req-696006cc-346f-43f6-8099-b5c7710615fd service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:48.762767 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-83152ffd-dcc4-4ff3-bafd-3a2118997865 req-696006cc-346f-43f6-8099-b5c7710615fd service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:48.776406 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-83152ffd-dcc4-4ff3-bafd-3a2118997865 req-696006cc-346f-43f6-8099-b5c7710615fd service nova] Creating event network-changed:306889a6-f7da-46ac-91cc-f0f0db7e57c3 for instance 9a075e00-151c-4ac4-b454-deea3de38ef7 on hv-cinder-maste Aug 21 01:30:48.782974 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-83152ffd-dcc4-4ff3-bafd-3a2118997865 req-696006cc-346f-43f6-8099-b5c7710615fd service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035832 Aug 21 01:30:48.784635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 350/714] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:30:49.063330 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-208b6223-7525-41e5-bc2c-67d40640e08f tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "DELETE /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 204 len: 0 microversion: 2.1 time: 0.432830 Aug 21 01:30:49.064424 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 365/715] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:30:48 2022] DELETE /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 0 bytes in 435 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:30:49.074083 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-704d5291-b45a-4932-94ab-14afa38dd93d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:49.075032 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-704d5291-b45a-4932-94ab-14afa38dd93d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:49.081897 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-704d5291-b45a-4932-94ab-14afa38dd93d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:49.082402 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-704d5291-b45a-4932-94ab-14afa38dd93d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:49.082958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-704d5291-b45a-4932-94ab-14afa38dd93d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:49.083797 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-704d5291-b45a-4932-94ab-14afa38dd93d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:49.084404 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-704d5291-b45a-4932-94ab-14afa38dd93d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:49.084938 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-704d5291-b45a-4932-94ab-14afa38dd93d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:49.193043 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8c780dcb-18e8-4f43-81b0-bdbe339db505 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:49.193740 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8c780dcb-18e8-4f43-81b0-bdbe339db505 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:49.201737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c780dcb-18e8-4f43-81b0-bdbe339db505 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:49.202252 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c780dcb-18e8-4f43-81b0-bdbe339db505 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:49.207169 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c780dcb-18e8-4f43-81b0-bdbe339db505 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:49.207169 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c780dcb-18e8-4f43-81b0-bdbe339db505 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:49.207169 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c780dcb-18e8-4f43-81b0-bdbe339db505 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:49.207169 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8c780dcb-18e8-4f43-81b0-bdbe339db505 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:49.328167 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-704d5291-b45a-4932-94ab-14afa38dd93d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:49.342357 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-704d5291-b45a-4932-94ab-14afa38dd93d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.273102 Aug 21 01:30:49.343488 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 351/716] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:49 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:49.396483 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8c780dcb-18e8-4f43-81b0-bdbe339db505 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:49.406183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8c780dcb-18e8-4f43-81b0-bdbe339db505 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1514 microversion: 2.1 time: 0.217032 Aug 21 01:30:49.406948 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 366/717] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:49 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1514 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:50.365889 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7d314a7c-1c13-4cbd-a1ce-627154a5a950 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:50.366534 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7d314a7c-1c13-4cbd-a1ce-627154a5a950 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:50.373912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7d314a7c-1c13-4cbd-a1ce-627154a5a950 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:50.374441 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7d314a7c-1c13-4cbd-a1ce-627154a5a950 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:50.374884 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7d314a7c-1c13-4cbd-a1ce-627154a5a950 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:50.375583 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7d314a7c-1c13-4cbd-a1ce-627154a5a950 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:50.376066 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7d314a7c-1c13-4cbd-a1ce-627154a5a950 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:50.376528 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7d314a7c-1c13-4cbd-a1ce-627154a5a950 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:50.438904 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-acc168fb-d46e-4a27-8b49-f056f061e5f0 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:50.439947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-acc168fb-d46e-4a27-8b49-f056f061e5f0 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:50.450926 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-acc168fb-d46e-4a27-8b49-f056f061e5f0 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:50.451603 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-acc168fb-d46e-4a27-8b49-f056f061e5f0 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:50.452175 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-acc168fb-d46e-4a27-8b49-f056f061e5f0 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:50.453018 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-acc168fb-d46e-4a27-8b49-f056f061e5f0 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:50.453592 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-acc168fb-d46e-4a27-8b49-f056f061e5f0 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:50.454130 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-acc168fb-d46e-4a27-8b49-f056f061e5f0 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:50.582099 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7d314a7c-1c13-4cbd-a1ce-627154a5a950 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:50.591417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7d314a7c-1c13-4cbd-a1ce-627154a5a950 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.229492 Aug 21 01:30:50.591995 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 352/718] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:50 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:50.630727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-acc168fb-d46e-4a27-8b49-f056f061e5f0 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:50.638401 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-acc168fb-d46e-4a27-8b49-f056f061e5f0 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1514 microversion: 2.1 time: 0.208109 Aug 21 01:30:50.638978 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 367/719] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:50 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1514 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:51.612405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-84d098d4-c348-47a8-afaf-8f291736a1d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:51.613157 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-84d098d4-c348-47a8-afaf-8f291736a1d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:51.619834 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-84d098d4-c348-47a8-afaf-8f291736a1d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:51.620365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-84d098d4-c348-47a8-afaf-8f291736a1d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:51.621431 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-84d098d4-c348-47a8-afaf-8f291736a1d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:51.622111 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-84d098d4-c348-47a8-afaf-8f291736a1d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:51.622560 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-84d098d4-c348-47a8-afaf-8f291736a1d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:51.622980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-84d098d4-c348-47a8-afaf-8f291736a1d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:51.658937 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-fee0f05a-dd3c-4cd2-87f3-fd634a6947e6 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:51.659888 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-fee0f05a-dd3c-4cd2-87f3-fd634a6947e6 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:51.666333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fee0f05a-dd3c-4cd2-87f3-fd634a6947e6 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:51.666837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fee0f05a-dd3c-4cd2-87f3-fd634a6947e6 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:51.667773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fee0f05a-dd3c-4cd2-87f3-fd634a6947e6 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:51.668511 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fee0f05a-dd3c-4cd2-87f3-fd634a6947e6 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:51.668986 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fee0f05a-dd3c-4cd2-87f3-fd634a6947e6 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:51.669413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fee0f05a-dd3c-4cd2-87f3-fd634a6947e6 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:51.813281 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-84d098d4-c348-47a8-afaf-8f291736a1d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:51.821768 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-84d098d4-c348-47a8-afaf-8f291736a1d3 tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.213180 Aug 21 01:30:51.822401 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 353/720] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:51 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:51.845200 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-fee0f05a-dd3c-4cd2-87f3-fd634a6947e6 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:51.856273 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-fee0f05a-dd3c-4cd2-87f3-fd634a6947e6 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1514 microversion: 2.1 time: 0.200867 Aug 21 01:30:51.856933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 368/721] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:51 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1514 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:52.843025 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-eb8194c0-af49-4461-a7fa-13d824631e7d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:52.843803 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-eb8194c0-af49-4461-a7fa-13d824631e7d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:52.850928 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb8194c0-af49-4461-a7fa-13d824631e7d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:52.851807 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb8194c0-af49-4461-a7fa-13d824631e7d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:52.852342 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb8194c0-af49-4461-a7fa-13d824631e7d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:52.853159 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb8194c0-af49-4461-a7fa-13d824631e7d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:52.853670 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb8194c0-af49-4461-a7fa-13d824631e7d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:52.854184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb8194c0-af49-4461-a7fa-13d824631e7d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:52.859579 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:52.883578 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8e3e3f20-7f6b-4958-9658-eaaefcdcc4e2 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:52.884873 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8e3e3f20-7f6b-4958-9658-eaaefcdcc4e2 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:52.894649 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e3e3f20-7f6b-4958-9658-eaaefcdcc4e2 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:52.895148 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e3e3f20-7f6b-4958-9658-eaaefcdcc4e2 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:52.895584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e3e3f20-7f6b-4958-9658-eaaefcdcc4e2 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:52.896501 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e3e3f20-7f6b-4958-9658-eaaefcdcc4e2 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:52.897040 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e3e3f20-7f6b-4958-9658-eaaefcdcc4e2 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:52.897476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e3e3f20-7f6b-4958-9658-eaaefcdcc4e2 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:53.276857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-eb8194c0-af49-4461-a7fa-13d824631e7d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:53.284804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-eb8194c0-af49-4461-a7fa-13d824631e7d tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1522 microversion: 2.1 time: 0.445643 Aug 21 01:30:53.285395 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 354/722] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:52 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1522 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:53.301734 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8e3e3f20-7f6b-4958-9658-eaaefcdcc4e2 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:53.312788 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8e3e3f20-7f6b-4958-9658-eaaefcdcc4e2 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1514 microversion: 2.1 time: 0.438291 Aug 21 01:30:53.327472 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 369/723] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:52 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1514 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:53.372277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-b49a2534-0c15-4571-9c62-953916717143 req-83e0355e-9bcd-447b-8216-7b378b28a62b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad", "name": "network-vif-deleted", "tag": "240f3a1c-670e-4e79-90ed-faa414cffbc0"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:53.382203 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-b49a2534-0c15-4571-9c62-953916717143 req-83e0355e-9bcd-447b-8216-7b378b28a62b service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:53.383011 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-b49a2534-0c15-4571-9c62-953916717143 req-83e0355e-9bcd-447b-8216-7b378b28a62b service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:53.383641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-b49a2534-0c15-4571-9c62-953916717143 req-83e0355e-9bcd-447b-8216-7b378b28a62b service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:53.399202 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-b49a2534-0c15-4571-9c62-953916717143 req-83e0355e-9bcd-447b-8216-7b378b28a62b service nova] Creating event network-vif-deleted:240f3a1c-670e-4e79-90ed-faa414cffbc0 for instance bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad on hv-cinder-maste Aug 21 01:30:53.406580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-b49a2534-0c15-4571-9c62-953916717143 req-83e0355e-9bcd-447b-8216-7b378b28a62b service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.119996 Aug 21 01:30:53.414981 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 355/724] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:53.731769 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-9885d4be-4adc-4e61-bde8-e7e90b632560 req-d69b9c04-6b33-450a-8718-a933b36557dc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "90e88110-1175-4ccf-bf6b-8efb43ab945e", "name": "network-vif-deleted", "tag": "f56acc10-3115-43fe-929a-cb8d30842aef"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:53.741477 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9885d4be-4adc-4e61-bde8-e7e90b632560 req-d69b9c04-6b33-450a-8718-a933b36557dc service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:53.741958 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9885d4be-4adc-4e61-bde8-e7e90b632560 req-d69b9c04-6b33-450a-8718-a933b36557dc service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:53.742395 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9885d4be-4adc-4e61-bde8-e7e90b632560 req-d69b9c04-6b33-450a-8718-a933b36557dc service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:53.754362 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-9885d4be-4adc-4e61-bde8-e7e90b632560 req-d69b9c04-6b33-450a-8718-a933b36557dc service nova] Creating event network-vif-deleted:f56acc10-3115-43fe-929a-cb8d30842aef for instance 90e88110-1175-4ccf-bf6b-8efb43ab945e on hv-cinder-maste Aug 21 01:30:53.759565 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-9885d4be-4adc-4e61-bde8-e7e90b632560 req-d69b9c04-6b33-450a-8718-a933b36557dc service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032097 Aug 21 01:30:53.760183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 370/725] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:54.307455 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:54.308050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:54.316458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:54.319897 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:54.319897 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:54.319897 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:54.319897 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:54.320352 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:54.335418 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:54.336163 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:54.345311 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:54.345819 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:54.346343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:54.347292 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:54.347917 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:54.348620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:54.355311 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:54.355694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:54.356170 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:54.356606 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lazy-loading 'fault' on Instance uuid 90e88110-1175-4ccf-bf6b-8efb43ab945e {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:30:54.385491 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:54.386012 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:54.387075 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:54.387574 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lazy-loading 'fault' on Instance uuid bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:30:54.411963 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd84b6d05decb47c7b55ac782d5839c1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9774c354b8b048768f2071acfb48a4bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:54.420057 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f7f46bce-d53b-4d92-9cbd-18577a00739a tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 200 len: 1312 microversion: 2.1 time: 0.116862 Aug 21 01:30:54.421091 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 356/726] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:54 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 1312 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:30:54.439677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2325a3d547d043d2a9ac95986187027b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '67f492f96d314cfea70bdbfa3caaee9e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:30:54.447499 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-173b7d63-0474-4549-a7c5-a5edcd58ca89 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 200 len: 1309 microversion: 2.1 time: 0.117112 Aug 21 01:30:54.448491 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 371/727] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:54 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 1309 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:30:55.443066 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c6340003-76de-4b58-8018-3cbd5b9f7eba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:55.443708 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c6340003-76de-4b58-8018-3cbd5b9f7eba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] [instance: 90e88110-1175-4ccf-bf6b-8efb43ab945e] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:55.451299 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c6340003-76de-4b58-8018-3cbd5b9f7eba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:55.451809 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c6340003-76de-4b58-8018-3cbd5b9f7eba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:55.452289 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c6340003-76de-4b58-8018-3cbd5b9f7eba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:55.452996 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c6340003-76de-4b58-8018-3cbd5b9f7eba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:55.453452 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c6340003-76de-4b58-8018-3cbd5b9f7eba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:55.453879 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c6340003-76de-4b58-8018-3cbd5b9f7eba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:55.469323 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-c6340003-76de-4b58-8018-3cbd5b9f7eba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] HTTP exception thrown: Instance 90e88110-1175-4ccf-bf6b-8efb43ab945e could not be found. Aug 21 01:30:55.470194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2737b27e-99b9-4c9f-bfd3-48157cd53a93 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c6340003-76de-4b58-8018-3cbd5b9f7eba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Request method failure captured: Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e HTTP/1.1 Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYl2grkk6r6DOR4WcdCDQaZ1LKSwt5ty8EvPKEsirHl25Bbz12fukVsuTZFRPo5OnAmJDHzJfldGu7OE-fBThLL0sUCHPYzsZbzZPojBGa0s89nUPxFfxXEogZCrYm2ZzOthYXvM-jM7ah9S1dz6CN84xJGSagaAUIMSBiUILa4u8mo Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 9774c354b8b048768f2071acfb48a4bf Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:30:55.471192 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:30:55.472724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/9774c354b8b048768f2071acfb48a4bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:30:55.472724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:30:55.472724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 9774c354b8b048768f2071acfb48a4bf Aug 21 01:30:55.472724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1204886246 Aug 21 01:30:55.472724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-ServerRescueNegativeTestJSON-1204886246-project Aug 21 01:30:55.472724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:30:55.472724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:30:55.472724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: d84b6d05decb47c7b55ac782d5839c1a Aug 21 01:30:55.472724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1204886246-project Aug 21 01:30:55.472724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:30:55.472724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 90e88110-1175-4ccf-bf6b-8efb43ab945e could not be found. Aug 21 01:30:55.472724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '90e88110-1175-4ccf-bf6b-8efb43ab945e'} Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 90e88110-1175-4ccf-bf6b-8efb43ab945e could not be found. Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:30:55.473532 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 90e88110-1175-4ccf-bf6b-8efb43ab945e could not be found. Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 90e88110-1175-4ccf-bf6b-8efb43ab945e could not be found. Aug 21 01:30:55.474983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:30:55.476737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-2737b27e-99b9-4c9f-bfd3-48157cd53a93 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] [instance: bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:55.476858 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c6340003-76de-4b58-8018-3cbd5b9f7eba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] Returning 404 to user: Instance 90e88110-1175-4ccf-bf6b-8efb43ab945e could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:30:55.476858 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c6340003-76de-4b58-8018-3cbd5b9f7eba tempest-ServerRescueNegativeTestJSON-1204886246 tempest-ServerRescueNegativeTestJSON-1204886246-project] 10.222.0.47 "GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e" status: 404 len: 111 microversion: 2.1 time: 0.033571 Aug 21 01:30:55.476858 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 357/728] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:55 2022] GET /compute/v2.1/servers/90e88110-1175-4ccf-bf6b-8efb43ab945e => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:30:55.485613 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2737b27e-99b9-4c9f-bfd3-48157cd53a93 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:55.486179 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2737b27e-99b9-4c9f-bfd3-48157cd53a93 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:55.486651 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2737b27e-99b9-4c9f-bfd3-48157cd53a93 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:55.487394 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2737b27e-99b9-4c9f-bfd3-48157cd53a93 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:55.487893 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2737b27e-99b9-4c9f-bfd3-48157cd53a93 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:55.488356 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2737b27e-99b9-4c9f-bfd3-48157cd53a93 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:55.502835 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-2737b27e-99b9-4c9f-bfd3-48157cd53a93 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] HTTP exception thrown: Instance bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad could not be found. Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2737b27e-99b9-4c9f-bfd3-48157cd53a93 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Request method failure captured: Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad HTTP/1.1 Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYpCUMdYFiBnjXzaa6PTyGyJsWfhcnlyjAzkjhPHAIWMJQEHncMLbhc8nFjmquAGwoV9oJnuJyERqn73N5Fn6fuoLFlkJv7tOQzLxl6tpfweEYS_LkReWnpfFGhQyNsYJu6J_MYOEXBiOWmTaYlit6pWtpxTXEYRlvUkjWkE6uNLBhg Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 67f492f96d314cfea70bdbfa3caaee9e Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1041524239 Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:30:55.504532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:30:55.506143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/67f492f96d314cfea70bdbfa3caaee9e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_67f492f96d314cfea70bdbfa3caaee9e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/67f492f96d314cfea70bdbfa3caaee9e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/67f492f96d314cfea70bdbfa3caaee9e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:30:55.506143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-VolumesSnapshotTestJSON-1041524239 Aug 21 01:30:55.506143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 67f492f96d314cfea70bdbfa3caaee9e Aug 21 01:30:55.506143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1041524239 Aug 21 01:30:55.506143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-VolumesSnapshotTestJSON-1041524239-project Aug 21 01:30:55.506143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:30:55.506143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:30:55.506143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 2325a3d547d043d2a9ac95986187027b Aug 21 01:30:55.506143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-VolumesSnapshotTestJSON-1041524239-project Aug 21 01:30:55.506143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:30:55.506143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad could not be found. Aug 21 01:30:55.506143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': 'bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad'} Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad could not be found. Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:30:55.507085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad could not be found. Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad could not be found. Aug 21 01:30:55.512516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:30:55.513876 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2737b27e-99b9-4c9f-bfd3-48157cd53a93 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] Returning 404 to user: Instance bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:30:55.513876 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2737b27e-99b9-4c9f-bfd3-48157cd53a93 tempest-VolumesSnapshotTestJSON-1041524239 tempest-VolumesSnapshotTestJSON-1041524239-project] 10.222.0.47 "GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad" status: 404 len: 111 microversion: 2.1 time: 0.041464 Aug 21 01:30:55.513876 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 372/729] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:30:55 2022] GET /compute/v2.1/servers/bbcc0aae-d0d3-4890-ba49-5ab2abbb13ad => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 21 01:30:56.451632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-0d03d527-53c8-4828-b65a-6997672ad824 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1575003272"}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:56.452253 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-0d03d527-53c8-4828-b65a-6997672ad824 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:56.459351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0d03d527-53c8-4828-b65a-6997672ad824 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:56.459850 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0d03d527-53c8-4828-b65a-6997672ad824 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:56.460386 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0d03d527-53c8-4828-b65a-6997672ad824 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:56.461335 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0d03d527-53c8-4828-b65a-6997672ad824 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:56.461804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0d03d527-53c8-4828-b65a-6997672ad824 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:56.462302 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0d03d527-53c8-4828-b65a-6997672ad824 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:56.689913 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.network.security_group_api [None req-0d03d527-53c8-4828-b65a-6997672ad824 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Removing security group 81fe8dd3-7ec5-4c04-b2b5-ba8178555d1c from port 9b282ab5-9500-4319-bbaa-ac3078e3128d Aug 21 01:30:57.268514 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-0d03d527-53c8-4828-b65a-6997672ad824 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "POST /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/action" status: 202 len: 0 microversion: 2.1 time: 0.820808 Aug 21 01:30:57.269292 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 358/730] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:30:56 2022] POST /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/action => generated 0 bytes in 822 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:30:57.304090 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-395f59d6-627f-4a9a-b669-4e9225df424f req-a18347a3-2000-4c10-b981-cafbbb98e65c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9bc908f3-d8e3-4432-8197-51096e59a470", "tag": "9b282ab5-9500-4319-bbaa-ac3078e3128d"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:57.326695 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-395f59d6-627f-4a9a-b669-4e9225df424f req-a18347a3-2000-4c10-b981-cafbbb98e65c service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:57.329789 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-395f59d6-627f-4a9a-b669-4e9225df424f req-a18347a3-2000-4c10-b981-cafbbb98e65c service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:57.330494 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-395f59d6-627f-4a9a-b669-4e9225df424f req-a18347a3-2000-4c10-b981-cafbbb98e65c service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:57.379390 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-395f59d6-627f-4a9a-b669-4e9225df424f req-a18347a3-2000-4c10-b981-cafbbb98e65c service nova] Creating event network-changed:9b282ab5-9500-4319-bbaa-ac3078e3128d for instance 9bc908f3-d8e3-4432-8197-51096e59a470 on hv-cinder-maste Aug 21 01:30:57.388803 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-395f59d6-627f-4a9a-b669-4e9225df424f req-a18347a3-2000-4c10-b981-cafbbb98e65c service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089114 Aug 21 01:30:57.390042 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 373/731] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:59.472403 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-ab1fa250-c7a0-4739-aeda-6d3bceb56be6 req-2fc2b5f4-29b2-4b6b-b381-ec595db99830 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9bc908f3-d8e3-4432-8197-51096e59a470", "tag": "9b282ab5-9500-4319-bbaa-ac3078e3128d"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:30:59.475139 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6b0848a2-7a03-45c8-9445-d245e2d30519 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:59.482935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6b0848a2-7a03-45c8-9445-d245e2d30519 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:59.485283 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=73,nova_api:DELETE=1,nova_api:UPDATE=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:30:59.492290 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6b0848a2-7a03-45c8-9445-d245e2d30519 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:59.493219 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6b0848a2-7a03-45c8-9445-d245e2d30519 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:59.493776 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6b0848a2-7a03-45c8-9445-d245e2d30519 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:59.494410 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6b0848a2-7a03-45c8-9445-d245e2d30519 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:59.494862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6b0848a2-7a03-45c8-9445-d245e2d30519 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:59.495284 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6b0848a2-7a03-45c8-9445-d245e2d30519 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:59.500240 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ab1fa250-c7a0-4739-aeda-6d3bceb56be6 req-2fc2b5f4-29b2-4b6b-b381-ec595db99830 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:59.500780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ab1fa250-c7a0-4739-aeda-6d3bceb56be6 req-2fc2b5f4-29b2-4b6b-b381-ec595db99830 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:59.501147 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ab1fa250-c7a0-4739-aeda-6d3bceb56be6 req-2fc2b5f4-29b2-4b6b-b381-ec595db99830 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:59.516302 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-ab1fa250-c7a0-4739-aeda-6d3bceb56be6 req-2fc2b5f4-29b2-4b6b-b381-ec595db99830 service nova] Creating event network-changed:9b282ab5-9500-4319-bbaa-ac3078e3128d for instance 9bc908f3-d8e3-4432-8197-51096e59a470 on hv-cinder-maste Aug 21 01:30:59.531730 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-ab1fa250-c7a0-4739-aeda-6d3bceb56be6 req-2fc2b5f4-29b2-4b6b-b381-ec595db99830 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063679 Aug 21 01:30:59.536554 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 374/732] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:30:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:30:59.629715 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-6b0848a2-7a03-45c8-9445-d245e2d30519 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lazy-loading 'services' on Instance uuid 9bc908f3-d8e3-4432-8197-51096e59a470 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:30:59.881074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6b0848a2-7a03-45c8-9445-d245e2d30519 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "DELETE /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments/15bddfef-2aaf-400f-80f6-4ea3ebd9c646" status: 202 len: 0 microversion: 2.1 time: 0.418420 Aug 21 01:30:59.881768 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 359/733] 10.222.0.47 () {58 vars in 1416 bytes} [Sun Aug 21 01:30:59 2022] DELETE /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments/15bddfef-2aaf-400f-80f6-4ea3ebd9c646 => generated 0 bytes in 420 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:30:59.890836 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7f2fec45-46a5-4c2a-96e0-e74f8c927ddc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:30:59.893428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7f2fec45-46a5-4c2a-96e0-e74f8c927ddc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:30:59.899999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f2fec45-46a5-4c2a-96e0-e74f8c927ddc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:59.900615 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f2fec45-46a5-4c2a-96e0-e74f8c927ddc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:59.901018 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f2fec45-46a5-4c2a-96e0-e74f8c927ddc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:59.901649 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f2fec45-46a5-4c2a-96e0-e74f8c927ddc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:30:59.902321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f2fec45-46a5-4c2a-96e0-e74f8c927ddc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:30:59.903015 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f2fec45-46a5-4c2a-96e0-e74f8c927ddc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:30:59.933001 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7f2fec45-46a5-4c2a-96e0-e74f8c927ddc tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046442 Aug 21 01:30:59.934183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 375/734] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:30:59 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:31:00.947097 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-5e3b33b4-d7b8-48f3-834f-cc5db9fdc8f8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:00.949815 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-5e3b33b4-d7b8-48f3-834f-cc5db9fdc8f8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:00.956641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5e3b33b4-d7b8-48f3-834f-cc5db9fdc8f8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:00.957209 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5e3b33b4-d7b8-48f3-834f-cc5db9fdc8f8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:00.957635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5e3b33b4-d7b8-48f3-834f-cc5db9fdc8f8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:00.958313 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5e3b33b4-d7b8-48f3-834f-cc5db9fdc8f8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:00.958762 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5e3b33b4-d7b8-48f3-834f-cc5db9fdc8f8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:00.959187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5e3b33b4-d7b8-48f3-834f-cc5db9fdc8f8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:00.994472 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5e3b33b4-d7b8-48f3-834f-cc5db9fdc8f8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052352 Aug 21 01:31:00.995054 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 360/735] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:31:00 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:02.007120 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-1e6e9594-2d3c-4be1-bf26-5c765791c3dd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:02.009375 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-1e6e9594-2d3c-4be1-bf26-5c765791c3dd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:02.017337 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1e6e9594-2d3c-4be1-bf26-5c765791c3dd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:02.017829 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1e6e9594-2d3c-4be1-bf26-5c765791c3dd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:02.018265 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1e6e9594-2d3c-4be1-bf26-5c765791c3dd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:02.018960 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1e6e9594-2d3c-4be1-bf26-5c765791c3dd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:02.019413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1e6e9594-2d3c-4be1-bf26-5c765791c3dd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:02.019839 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1e6e9594-2d3c-4be1-bf26-5c765791c3dd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:02.074609 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-1e6e9594-2d3c-4be1-bf26-5c765791c3dd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071435 Aug 21 01:31:02.075198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 376/736] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:31:02 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:03.093691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9ba54e38-4738-4764-be23-91d2f1e96116 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:03.099783 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9ba54e38-4738-4764-be23-91d2f1e96116 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:03.107452 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ba54e38-4738-4764-be23-91d2f1e96116 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:03.107979 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ba54e38-4738-4764-be23-91d2f1e96116 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:03.108432 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ba54e38-4738-4764-be23-91d2f1e96116 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:03.109286 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ba54e38-4738-4764-be23-91d2f1e96116 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:03.109966 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ba54e38-4738-4764-be23-91d2f1e96116 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:03.110267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9ba54e38-4738-4764-be23-91d2f1e96116 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:03.140250 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9ba54e38-4738-4764-be23-91d2f1e96116 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.055721 Aug 21 01:31:03.140841 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 361/737] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:31:03 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments => generated 25 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 21 01:31:03.151582 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-68b6373c-c63b-41c1-b6e4-543eae089fb0 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:03.152786 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-68b6373c-c63b-41c1-b6e4-543eae089fb0 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:03.159558 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68b6373c-c63b-41c1-b6e4-543eae089fb0 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:03.160051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68b6373c-c63b-41c1-b6e4-543eae089fb0 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:03.160510 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68b6373c-c63b-41c1-b6e4-543eae089fb0 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:03.161211 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68b6373c-c63b-41c1-b6e4-543eae089fb0 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:03.161665 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68b6373c-c63b-41c1-b6e4-543eae089fb0 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:03.162090 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68b6373c-c63b-41c1-b6e4-543eae089fb0 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:03.242091 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-68b6373c-c63b-41c1-b6e4-543eae089fb0 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] HTTP exception thrown: Instance 9bc908f3-d8e3-4432-8197-51096e59a470 is not attached to volume 15bddfef-2aaf-400f-80f6-4ea3ebd9c646 Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-68b6373c-c63b-41c1-b6e4-543eae089fb0 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Request method failure captured: Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: DELETE /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments/15bddfef-2aaf-400f-80f6-4ea3ebd9c646 HTTP/1.1 Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYoBeaVsL8XrQ4fNEkuNcx0hAORTElGVmksMxkczpgxLF8pC4IrmG-ftaqwYjQjLxWAie7nOHkIRHl99pHfAl8UImU9sm61vZtxivu4mS2W3rrfQiYJxTqEglazEHGZgtHTzIBYuwxvkq5utHaByelgA0slaZjyOpP7_61FzctP_yUo Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: da8fc458ef6b49e18daac9fb195a2149 Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-TestMinimumBasicScenario-1908485893 Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:31:03.246823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:31:03.248497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/da8fc458ef6b49e18daac9fb195a2149"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_da8fc458ef6b49e18daac9fb195a2149"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/da8fc458ef6b49e18daac9fb195a2149"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/da8fc458ef6b49e18daac9fb195a2149"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:31:03.248497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-TestMinimumBasicScenario-1908485893 Aug 21 01:31:03.248497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: da8fc458ef6b49e18daac9fb195a2149 Aug 21 01:31:03.248497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1908485893 Aug 21 01:31:03.248497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-TestMinimumBasicScenario-1908485893-project Aug 21 01:31:03.248497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:31:03.248497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:31:03.248497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 2a6687a3b09445d897879095287a8586 Aug 21 01:31:03.248497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-TestMinimumBasicScenario-1908485893-project Aug 21 01:31:03.248497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:31:03.248497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 9bc908f3-d8e3-4432-8197-51096e59a470 is not attached to volume 15bddfef-2aaf-400f-80f6-4ea3ebd9c646 Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'server_id': '9bc908f3-d8e3-4432-8197-51096e59a470', 'id': '15bddfef-2aaf-400f-80f6-4ea3ebd9c646'} Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 15bddfef-2aaf-400f-80f6-4ea3ebd9c646. Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:31:03.249307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9bc908f3-d8e3-4432-8197-51096e59a470 is not attached to volume 15bddfef-2aaf-400f-80f6-4ea3ebd9c646 Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9bc908f3-d8e3-4432-8197-51096e59a470 is not attached to volume 15bddfef-2aaf-400f-80f6-4ea3ebd9c646 Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:31:03.250748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-68b6373c-c63b-41c1-b6e4-543eae089fb0 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Returning 404 to user: Instance 9bc908f3-d8e3-4432-8197-51096e59a470 is not attached to volume 15bddfef-2aaf-400f-80f6-4ea3ebd9c646 {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:31:03.251729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-68b6373c-c63b-41c1-b6e4-543eae089fb0 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "DELETE /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments/15bddfef-2aaf-400f-80f6-4ea3ebd9c646" status: 404 len: 154 microversion: 2.1 time: 0.101748 Aug 21 01:31:03.251729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 377/738] 10.222.0.47 () {58 vars in 1416 bytes} [Sun Aug 21 01:31:03 2022] DELETE /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470/os-volume_attachments/15bddfef-2aaf-400f-80f6-4ea3ebd9c646 => generated 154 bytes in 104 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 21 01:31:04.623652 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f4ccccc5-9557-4db3-89f2-dc559aca6eae tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:04.624404 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f4ccccc5-9557-4db3-89f2-dc559aca6eae tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:04.632568 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4ccccc5-9557-4db3-89f2-dc559aca6eae tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:04.633083 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4ccccc5-9557-4db3-89f2-dc559aca6eae tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:04.633517 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4ccccc5-9557-4db3-89f2-dc559aca6eae tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:04.634290 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4ccccc5-9557-4db3-89f2-dc559aca6eae tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:04.634805 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4ccccc5-9557-4db3-89f2-dc559aca6eae tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:04.635360 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4ccccc5-9557-4db3-89f2-dc559aca6eae tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:04.667782 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f4ccccc5-9557-4db3-89f2-dc559aca6eae tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:31:05.086697 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f4ccccc5-9557-4db3-89f2-dc559aca6eae tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "DELETE /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 204 len: 0 microversion: 2.1 time: 0.466787 Aug 21 01:31:05.087293 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 362/739] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:31:04 2022] DELETE /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:31:05.098091 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-028cb83d-f6a1-4ec5-ab99-6a6b472df648 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:05.098091 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-028cb83d-f6a1-4ec5-ab99-6a6b472df648 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:05.105265 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-028cb83d-f6a1-4ec5-ab99-6a6b472df648 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:05.105773 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-028cb83d-f6a1-4ec5-ab99-6a6b472df648 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:05.106208 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-028cb83d-f6a1-4ec5-ab99-6a6b472df648 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:05.106902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-028cb83d-f6a1-4ec5-ab99-6a6b472df648 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:05.107405 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-028cb83d-f6a1-4ec5-ab99-6a6b472df648 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:05.107827 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-028cb83d-f6a1-4ec5-ab99-6a6b472df648 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:05.113407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:31:05.250951 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-028cb83d-f6a1-4ec5-ab99-6a6b472df648 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:05.266884 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-028cb83d-f6a1-4ec5-ab99-6a6b472df648 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1554 microversion: 2.1 time: 0.173843 Aug 21 01:31:05.267779 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 378/740] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:05 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1554 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:06.294462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6807043b-7334-4741-ad3e-3d6de0c7f850 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:06.295233 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6807043b-7334-4741-ad3e-3d6de0c7f850 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:06.302375 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6807043b-7334-4741-ad3e-3d6de0c7f850 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:06.303065 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6807043b-7334-4741-ad3e-3d6de0c7f850 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:06.303609 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6807043b-7334-4741-ad3e-3d6de0c7f850 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:06.304579 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6807043b-7334-4741-ad3e-3d6de0c7f850 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:06.305148 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6807043b-7334-4741-ad3e-3d6de0c7f850 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:06.305672 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6807043b-7334-4741-ad3e-3d6de0c7f850 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:06.501150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6807043b-7334-4741-ad3e-3d6de0c7f850 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:06.511096 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6807043b-7334-4741-ad3e-3d6de0c7f850 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1554 microversion: 2.1 time: 0.221700 Aug 21 01:31:06.515567 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 363/741] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:06 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1554 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:07.536141 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9f603ce2-e76c-486d-837f-cb4dd98c2d20 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:07.537419 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9f603ce2-e76c-486d-837f-cb4dd98c2d20 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:07.546032 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9f603ce2-e76c-486d-837f-cb4dd98c2d20 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:07.550116 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9f603ce2-e76c-486d-837f-cb4dd98c2d20 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:07.550749 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9f603ce2-e76c-486d-837f-cb4dd98c2d20 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:07.551716 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9f603ce2-e76c-486d-837f-cb4dd98c2d20 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:07.552440 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9f603ce2-e76c-486d-837f-cb4dd98c2d20 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:07.558010 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9f603ce2-e76c-486d-837f-cb4dd98c2d20 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:07.803067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9f603ce2-e76c-486d-837f-cb4dd98c2d20 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:07.811632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9f603ce2-e76c-486d-837f-cb4dd98c2d20 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1554 microversion: 2.1 time: 0.279254 Aug 21 01:31:07.812655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 379/742] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:07 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1554 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:08.185815 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-947eeb05-b762-4d31-88b8-d509a5deb316 req-149d3742-1f85-457d-b804-52e94969002a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9bc908f3-d8e3-4432-8197-51096e59a470", "name": "network-vif-deleted", "tag": "9b282ab5-9500-4319-bbaa-ac3078e3128d"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:08.197678 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-947eeb05-b762-4d31-88b8-d509a5deb316 req-149d3742-1f85-457d-b804-52e94969002a service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:08.198188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-947eeb05-b762-4d31-88b8-d509a5deb316 req-149d3742-1f85-457d-b804-52e94969002a service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:08.199686 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-947eeb05-b762-4d31-88b8-d509a5deb316 req-149d3742-1f85-457d-b804-52e94969002a service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:08.212977 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-947eeb05-b762-4d31-88b8-d509a5deb316 req-149d3742-1f85-457d-b804-52e94969002a service nova] Creating event network-vif-deleted:9b282ab5-9500-4319-bbaa-ac3078e3128d for instance 9bc908f3-d8e3-4432-8197-51096e59a470 on hv-cinder-maste Aug 21 01:31:08.222073 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-947eeb05-b762-4d31-88b8-d509a5deb316 req-149d3742-1f85-457d-b804-52e94969002a service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039916 Aug 21 01:31:08.222687 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 364/743] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:31:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:08.834861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-682912b6-9dec-4c35-91e5-bc736a9b37bd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:08.835492 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-682912b6-9dec-4c35-91e5-bc736a9b37bd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:08.855080 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-682912b6-9dec-4c35-91e5-bc736a9b37bd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:08.857172 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-682912b6-9dec-4c35-91e5-bc736a9b37bd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:08.861044 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-682912b6-9dec-4c35-91e5-bc736a9b37bd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:08.861808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-682912b6-9dec-4c35-91e5-bc736a9b37bd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:08.862283 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-682912b6-9dec-4c35-91e5-bc736a9b37bd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:08.862729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-682912b6-9dec-4c35-91e5-bc736a9b37bd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:08.953464 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-682912b6-9dec-4c35-91e5-bc736a9b37bd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:08.963229 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-682912b6-9dec-4c35-91e5-bc736a9b37bd tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1512 microversion: 2.1 time: 0.132088 Aug 21 01:31:08.964002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 380/744] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:08 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1512 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:09.255919 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6e636a6b-44e6-454a-ab06-1a2c72c8d16c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: .version_select at 0x7f9da33bdb80>, body: {"keypair": {"name": "tempest-keypair-1630789153", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBADPPzmQQ/x+bxEBakeo4HkhOay7ZRrVTOixHaebGYF9CPISgizmOSRH0evST7+MWnGZkwlI5DUm5T+Dx9064e4BpbzXImmLq04gm+nZoM8CB8fvE6s6YTbRNi535tZntg=="}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:09.272553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-6e636a6b-44e6-454a-ab06-1a2c72c8d16c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for project 122b46d9e29843a88c7ca5f019426539. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:31:09.278895 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-6e636a6b-44e6-454a-ab06-1a2c72c8d16c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for user a91ac0ba7e8c4418930852b0fa184247 and project 122b46d9e29843a88c7ca5f019426539. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:31:09.372276 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6e636a6b-44e6-454a-ab06-1a2c72c8d16c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.243690 Aug 21 01:31:09.373024 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 365/745] 10.222.0.47 () {60 vars in 1159 bytes} [Sun Aug 21 01:31:09 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:09.985848 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8ac69a65-f9f8-447e-9167-a72a7c22dad8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:09.986582 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8ac69a65-f9f8-447e-9167-a72a7c22dad8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:09.992920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8ac69a65-f9f8-447e-9167-a72a7c22dad8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:09.993400 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8ac69a65-f9f8-447e-9167-a72a7c22dad8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:09.993838 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8ac69a65-f9f8-447e-9167-a72a7c22dad8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:09.994528 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8ac69a65-f9f8-447e-9167-a72a7c22dad8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:09.994981 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8ac69a65-f9f8-447e-9167-a72a7c22dad8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:09.995402 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8ac69a65-f9f8-447e-9167-a72a7c22dad8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:10.063164 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8ac69a65-f9f8-447e-9167-a72a7c22dad8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:10.072672 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8ac69a65-f9f8-447e-9167-a72a7c22dad8 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1512 microversion: 2.1 time: 0.090439 Aug 21 01:31:10.088552 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 381/746] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:09 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1512 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:11.100410 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-41931d10-a766-4fe8-bb19-a0869d45dd65 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:11.101969 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-41931d10-a766-4fe8-bb19-a0869d45dd65 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:11.114972 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41931d10-a766-4fe8-bb19-a0869d45dd65 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:11.114972 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41931d10-a766-4fe8-bb19-a0869d45dd65 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:11.114972 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41931d10-a766-4fe8-bb19-a0869d45dd65 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:11.114972 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41931d10-a766-4fe8-bb19-a0869d45dd65 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:11.115793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41931d10-a766-4fe8-bb19-a0869d45dd65 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:11.115793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-41931d10-a766-4fe8-bb19-a0869d45dd65 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:11.207516 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-41931d10-a766-4fe8-bb19-a0869d45dd65 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:11.218602 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-41931d10-a766-4fe8-bb19-a0869d45dd65 tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1512 microversion: 2.1 time: 0.124267 Aug 21 01:31:11.223454 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 366/747] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:11 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1512 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:11.847488 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-690195937", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "99061e98-94a6-4cdf-b3c1-c9af69a6346c"}], "security_groups": [{"name": "tempest-securitygroup--745825344"}], "key_name": "tempest-keypair-1630789153", "user_data": "}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:12.000727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.network.neutron [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] validate_networks() for [('99061e98-94a6-4cdf-b3c1-c9af69a6346c', None, None, None, None, None)] {{(pid=121277) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:31:12.257731 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:12.263800 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:12.275094 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:12.275922 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:12.276421 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:12.283007 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:12.283600 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:12.285175 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:12.356493 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:12.357902 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:12.358600 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:12.359240 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lazy-loading 'fault' on Instance uuid 9bc908f3-d8e3-4432-8197-51096e59a470 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:31:12.454549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2a6687a3b09445d897879095287a8586', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da8fc458ef6b49e18daac9fb195a2149', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:12.473446 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d5e883cb-80c9-4d9e-89c8-2406b8326d8a tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 200 len: 1347 microversion: 2.1 time: 0.233056 Aug 21 01:31:12.474302 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 367/748] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:12 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 1347 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:12.585259 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Flavor limits 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:31:12.585698 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Image limits 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:31:12.586533 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Flavor pref 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:31:12.587173 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Image pref 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:31:12.587822 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:31:12.598291 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:12.598843 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:12.599279 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:12.600149 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:12.600618 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:12.601046 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:12.621845 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for project 122b46d9e29843a88c7ca5f019426539. Resources: {'instances', 'ram', 'cores'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:31:12.629291 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for user a91ac0ba7e8c4418930852b0fa184247 and project 122b46d9e29843a88c7ca5f019426539. Resources: {'instances', 'ram', 'cores'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:31:12.647607 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Going to run 1 instances... {{(pid=121277) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:31:12.650448 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121277) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:31:12.815811 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5312e067-5265-4caa-9a7a-c6bf98892806 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.977076 Aug 21 01:31:12.831821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 382/749] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:31:11 2022] POST /compute/v2.1/servers => generated 401 bytes in 980 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:31:12.836846 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-956e3793-e726-4583-bf6c-dcd293c253f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:12.837469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-956e3793-e726-4583-bf6c-dcd293c253f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:12.904662 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-956e3793-e726-4583-bf6c-dcd293c253f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:12.913792 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-956e3793-e726-4583-bf6c-dcd293c253f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1241 microversion: 2.1 time: 0.080706 Aug 21 01:31:12.948559 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 368/750] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:12 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1241 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:13.499092 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6738a5cb-a9e0-45e3-8985-e10778ecb5cf tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:13.499827 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6738a5cb-a9e0-45e3-8985-e10778ecb5cf tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] [instance: 9bc908f3-d8e3-4432-8197-51096e59a470] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:13.506918 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6738a5cb-a9e0-45e3-8985-e10778ecb5cf tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:13.507433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6738a5cb-a9e0-45e3-8985-e10778ecb5cf tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:13.507906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6738a5cb-a9e0-45e3-8985-e10778ecb5cf tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:13.508708 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6738a5cb-a9e0-45e3-8985-e10778ecb5cf tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:13.509235 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6738a5cb-a9e0-45e3-8985-e10778ecb5cf tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:13.509670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6738a5cb-a9e0-45e3-8985-e10778ecb5cf tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:13.524826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-6738a5cb-a9e0-45e3-8985-e10778ecb5cf tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] HTTP exception thrown: Instance 9bc908f3-d8e3-4432-8197-51096e59a470 could not be found. Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6738a5cb-a9e0-45e3-8985-e10778ecb5cf tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Request method failure captured: Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 HTTP/1.1 Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYoBeaVsL8XrQ4fNEkuNcx0hAORTElGVmksMxkczpgxLF8pC4IrmG-ftaqwYjQjLxWAie7nOHkIRHl99pHfAl8UImU9sm61vZtxivu4mS2W3rrfQiYJxTqEglazEHGZgtHTzIBYuwxvkq5utHaByelgA0slaZjyOpP7_61FzctP_yUo Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: da8fc458ef6b49e18daac9fb195a2149 Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-TestMinimumBasicScenario-1908485893 Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:31:13.526467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:31:13.530062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/da8fc458ef6b49e18daac9fb195a2149"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_da8fc458ef6b49e18daac9fb195a2149"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/da8fc458ef6b49e18daac9fb195a2149"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/da8fc458ef6b49e18daac9fb195a2149"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:31:13.530062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-TestMinimumBasicScenario-1908485893 Aug 21 01:31:13.530062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: da8fc458ef6b49e18daac9fb195a2149 Aug 21 01:31:13.530062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1908485893 Aug 21 01:31:13.530062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-TestMinimumBasicScenario-1908485893-project Aug 21 01:31:13.530062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:31:13.530062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:31:13.530062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 2a6687a3b09445d897879095287a8586 Aug 21 01:31:13.530062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-TestMinimumBasicScenario-1908485893-project Aug 21 01:31:13.530062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:31:13.530062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 9bc908f3-d8e3-4432-8197-51096e59a470 could not be found. Aug 21 01:31:13.530062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '9bc908f3-d8e3-4432-8197-51096e59a470'} Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9bc908f3-d8e3-4432-8197-51096e59a470 could not be found. Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:31:13.530987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9bc908f3-d8e3-4432-8197-51096e59a470 could not be found. Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9bc908f3-d8e3-4432-8197-51096e59a470 could not be found. Aug 21 01:31:13.532766 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:31:13.534138 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6738a5cb-a9e0-45e3-8985-e10778ecb5cf tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Returning 404 to user: Instance 9bc908f3-d8e3-4432-8197-51096e59a470 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:31:13.534138 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6738a5cb-a9e0-45e3-8985-e10778ecb5cf tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470" status: 404 len: 111 microversion: 2.1 time: 0.034659 Aug 21 01:31:13.534138 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 383/751] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:13 2022] GET /compute/v2.1/servers/9bc908f3-d8e3-4432-8197-51096e59a470 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:31:13.536967 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d8821e43-6bc6-4cae-819d-783d815158ed tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] Calling method '.version_select at 0x7f9da33650d0>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:13.613822 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d8821e43-6bc6-4cae-819d-783d815158ed tempest-TestMinimumBasicScenario-1908485893 tempest-TestMinimumBasicScenario-1908485893-project] 10.222.0.47 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1001539393" status: 202 len: 0 microversion: 2.1 time: 0.080564 Aug 21 01:31:13.614395 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 369/752] 10.222.0.47 () {58 vars in 1272 bytes} [Sun Aug 21 01:31:13 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1001539393 => generated 0 bytes in 82 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:31:13.939663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a11af9c0-1040-4314-97e4-e6315f808e78 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:13.940404 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a11af9c0-1040-4314-97e4-e6315f808e78 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:14.004024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a11af9c0-1040-4314-97e4-e6315f808e78 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:14.010994 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a11af9c0-1040-4314-97e4-e6315f808e78 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1241 microversion: 2.1 time: 0.075165 Aug 21 01:31:14.011557 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 384/753] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:13 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1241 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:14.676858 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d469efac-266b-44b5-918c-e2587f5352b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:14.677469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d469efac-266b-44b5-918c-e2587f5352b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:14.683409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d469efac-266b-44b5-918c-e2587f5352b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:14.684610 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d469efac-266b-44b5-918c-e2587f5352b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:14.685171 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d469efac-266b-44b5-918c-e2587f5352b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:14.686001 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d469efac-266b-44b5-918c-e2587f5352b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:14.686552 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d469efac-266b-44b5-918c-e2587f5352b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:14.689638 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d469efac-266b-44b5-918c-e2587f5352b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:14.712125 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d469efac-266b-44b5-918c-e2587f5352b8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/ips" status: 200 len: 150 microversion: 2.20 time: 0.039906 Aug 21 01:31:14.712749 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 370/754] 10.222.0.47 () {60 vars in 1289 bytes} [Sun Aug 21 01:31:14 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/ips => generated 150 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 21 01:31:15.031500 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d5009569-8732-4438-a43c-0283e7037be2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:15.032149 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d5009569-8732-4438-a43c-0283e7037be2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:15.039323 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d5009569-8732-4438-a43c-0283e7037be2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:15.039824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d5009569-8732-4438-a43c-0283e7037be2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:15.040307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d5009569-8732-4438-a43c-0283e7037be2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:15.041032 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d5009569-8732-4438-a43c-0283e7037be2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:15.041542 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d5009569-8732-4438-a43c-0283e7037be2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:15.041939 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d5009569-8732-4438-a43c-0283e7037be2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:15.108405 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d5009569-8732-4438-a43c-0283e7037be2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:15.116249 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d5009569-8732-4438-a43c-0283e7037be2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1293 microversion: 2.1 time: 0.088684 Aug 21 01:31:15.116991 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 385/755] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:15 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1293 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:16.138332 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-649b4017-ff37-4d28-8548-fde88ffc9051 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:16.138951 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-649b4017-ff37-4d28-8548-fde88ffc9051 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:16.147533 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-649b4017-ff37-4d28-8548-fde88ffc9051 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:16.148154 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-649b4017-ff37-4d28-8548-fde88ffc9051 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:16.148851 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-649b4017-ff37-4d28-8548-fde88ffc9051 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:16.149778 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-649b4017-ff37-4d28-8548-fde88ffc9051 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:16.150245 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-649b4017-ff37-4d28-8548-fde88ffc9051 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:16.150848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-649b4017-ff37-4d28-8548-fde88ffc9051 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:16.395401 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-649b4017-ff37-4d28-8548-fde88ffc9051 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:16.413369 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-649b4017-ff37-4d28-8548-fde88ffc9051 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1301 microversion: 2.1 time: 0.279227 Aug 21 01:31:16.413976 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 371/756] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:16 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1301 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:17.441050 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e2f05eb4-2938-4d5f-a20f-d3f994d8347a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:17.445107 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e2f05eb4-2938-4d5f-a20f-d3f994d8347a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:17.449062 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e2f05eb4-2938-4d5f-a20f-d3f994d8347a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:17.449607 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e2f05eb4-2938-4d5f-a20f-d3f994d8347a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:17.450377 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e2f05eb4-2938-4d5f-a20f-d3f994d8347a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:17.451099 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e2f05eb4-2938-4d5f-a20f-d3f994d8347a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:17.451528 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e2f05eb4-2938-4d5f-a20f-d3f994d8347a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:17.451954 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e2f05eb4-2938-4d5f-a20f-d3f994d8347a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:17.690952 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e2f05eb4-2938-4d5f-a20f-d3f994d8347a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:17.703834 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e2f05eb4-2938-4d5f-a20f-d3f994d8347a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1366 microversion: 2.1 time: 0.266513 Aug 21 01:31:17.704618 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 386/757] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:17 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1366 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:17.751403 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4639b9e7-71b2-4d45-8fe4-9eb55dfe2561 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:17.752781 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4639b9e7-71b2-4d45-8fe4-9eb55dfe2561 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:17.763351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4639b9e7-71b2-4d45-8fe4-9eb55dfe2561 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:17.764354 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4639b9e7-71b2-4d45-8fe4-9eb55dfe2561 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:17.765633 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4639b9e7-71b2-4d45-8fe4-9eb55dfe2561 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:17.766752 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4639b9e7-71b2-4d45-8fe4-9eb55dfe2561 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:17.767405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4639b9e7-71b2-4d45-8fe4-9eb55dfe2561 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:17.768023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4639b9e7-71b2-4d45-8fe4-9eb55dfe2561 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:17.805236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-4639b9e7-71b2-4d45-8fe4-9eb55dfe2561 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lazy-loading 'flavor' on Instance uuid a5b475aa-486a-4957-8b54-aa1ee7438a8a {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:31:18.014836 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:31:18.454033 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-50e0e7c5-3ea0-4557-bf54-96b55d5a5a9b req-317840ab-a9e1-4a81-8268-ba819652856d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1a9e859-eddf-4f45-8e02-168201b29a63", "tag": "316e8409-378a-47c3-a927-8b6aea8ed8bd"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:18.464895 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-50e0e7c5-3ea0-4557-bf54-96b55d5a5a9b req-317840ab-a9e1-4a81-8268-ba819652856d service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:18.465383 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-50e0e7c5-3ea0-4557-bf54-96b55d5a5a9b req-317840ab-a9e1-4a81-8268-ba819652856d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:18.465811 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-50e0e7c5-3ea0-4557-bf54-96b55d5a5a9b req-317840ab-a9e1-4a81-8268-ba819652856d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:18.479484 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-50e0e7c5-3ea0-4557-bf54-96b55d5a5a9b req-317840ab-a9e1-4a81-8268-ba819652856d service nova] Creating event network-changed:316e8409-378a-47c3-a927-8b6aea8ed8bd for instance d1a9e859-eddf-4f45-8e02-168201b29a63 on hv-cinder-maste Aug 21 01:31:18.484831 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-50e0e7c5-3ea0-4557-bf54-96b55d5a5a9b req-317840ab-a9e1-4a81-8268-ba819652856d service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035387 Aug 21 01:31:18.485460 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 387/758] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:31:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:18.546091 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4639b9e7-71b2-4d45-8fe4-9eb55dfe2561 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "POST /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/action" status: 202 len: 0 microversion: 2.20 time: 0.800184 Aug 21 01:31:18.546745 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 372/759] 10.222.0.47 () {62 vars in 1319 bytes} [Sun Aug 21 01:31:17 2022] POST /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/action => generated 0 bytes in 802 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:31:18.555467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-36d67d49-ae0f-44ed-b95d-b1cef86b4a4a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:18.556300 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-36d67d49-ae0f-44ed-b95d-b1cef86b4a4a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:18.564360 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-36d67d49-ae0f-44ed-b95d-b1cef86b4a4a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:18.565256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-36d67d49-ae0f-44ed-b95d-b1cef86b4a4a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:18.565783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-36d67d49-ae0f-44ed-b95d-b1cef86b4a4a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:18.566599 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-36d67d49-ae0f-44ed-b95d-b1cef86b4a4a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:18.567051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-36d67d49-ae0f-44ed-b95d-b1cef86b4a4a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:18.568122 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-36d67d49-ae0f-44ed-b95d-b1cef86b4a4a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:18.731637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-12c77cfb-b2d0-4ba1-aac3-e1f6c2e413c8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:18.733655 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-12c77cfb-b2d0-4ba1-aac3-e1f6c2e413c8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:18.741881 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-12c77cfb-b2d0-4ba1-aac3-e1f6c2e413c8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:18.742683 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-12c77cfb-b2d0-4ba1-aac3-e1f6c2e413c8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:18.743713 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-12c77cfb-b2d0-4ba1-aac3-e1f6c2e413c8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:18.749101 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-12c77cfb-b2d0-4ba1-aac3-e1f6c2e413c8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:18.749101 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-12c77cfb-b2d0-4ba1-aac3-e1f6c2e413c8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:18.749101 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-12c77cfb-b2d0-4ba1-aac3-e1f6c2e413c8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:18.821992 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-36d67d49-ae0f-44ed-b95d-b1cef86b4a4a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:18.833161 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-36d67d49-ae0f-44ed-b95d-b1cef86b4a4a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:18.835391 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-36d67d49-ae0f-44ed-b95d-b1cef86b4a4a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:18.840872 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-36d67d49-ae0f-44ed-b95d-b1cef86b4a4a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1720 microversion: 2.20 time: 0.285443 Aug 21 01:31:18.840872 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 388/760] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:18 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1720 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:31:18.990489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-12c77cfb-b2d0-4ba1-aac3-e1f6c2e413c8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:19.002529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-12c77cfb-b2d0-4ba1-aac3-e1f6c2e413c8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1366 microversion: 2.1 time: 0.272258 Aug 21 01:31:19.003594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 373/761] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:18 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1366 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:19.860440 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a842b3f9-3a3f-4c73-84fe-d3384cf8beeb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:19.861145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a842b3f9-3a3f-4c73-84fe-d3384cf8beeb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:19.863777 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=70,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:31:19.868877 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a842b3f9-3a3f-4c73-84fe-d3384cf8beeb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:19.869695 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a842b3f9-3a3f-4c73-84fe-d3384cf8beeb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:19.870217 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a842b3f9-3a3f-4c73-84fe-d3384cf8beeb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:19.872417 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a842b3f9-3a3f-4c73-84fe-d3384cf8beeb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:19.873675 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a842b3f9-3a3f-4c73-84fe-d3384cf8beeb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:19.874283 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a842b3f9-3a3f-4c73-84fe-d3384cf8beeb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:20.033549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-65840460-f81f-49a7-a463-a33d58d7fe5b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:20.034231 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-65840460-f81f-49a7-a463-a33d58d7fe5b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:20.042996 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65840460-f81f-49a7-a463-a33d58d7fe5b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:20.044044 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65840460-f81f-49a7-a463-a33d58d7fe5b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:20.044440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65840460-f81f-49a7-a463-a33d58d7fe5b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:20.045212 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65840460-f81f-49a7-a463-a33d58d7fe5b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:20.046648 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65840460-f81f-49a7-a463-a33d58d7fe5b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:20.047729 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-65840460-f81f-49a7-a463-a33d58d7fe5b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:20.137641 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a842b3f9-3a3f-4c73-84fe-d3384cf8beeb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:20.151010 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a842b3f9-3a3f-4c73-84fe-d3384cf8beeb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:20.152847 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a842b3f9-3a3f-4c73-84fe-d3384cf8beeb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:20.154199 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a842b3f9-3a3f-4c73-84fe-d3384cf8beeb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1720 microversion: 2.20 time: 0.297601 Aug 21 01:31:20.155170 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 389/762] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:19 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1720 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:20.279757 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-65840460-f81f-49a7-a463-a33d58d7fe5b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:20.288376 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-65840460-f81f-49a7-a463-a33d58d7fe5b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1366 microversion: 2.1 time: 0.259727 Aug 21 01:31:20.289121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 374/763] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:20 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1366 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:21.182376 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-606e21c1-f47c-4a5a-8623-1298fe6bb947 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:21.183202 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-606e21c1-f47c-4a5a-8623-1298fe6bb947 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:21.191287 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-606e21c1-f47c-4a5a-8623-1298fe6bb947 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:21.191975 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-606e21c1-f47c-4a5a-8623-1298fe6bb947 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:21.192575 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-606e21c1-f47c-4a5a-8623-1298fe6bb947 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:21.193662 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-606e21c1-f47c-4a5a-8623-1298fe6bb947 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:21.194131 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-606e21c1-f47c-4a5a-8623-1298fe6bb947 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:21.194758 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-606e21c1-f47c-4a5a-8623-1298fe6bb947 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:21.310355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1e772b1d-bf93-47fd-89d9-b3bda7421a67 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:21.310984 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1e772b1d-bf93-47fd-89d9-b3bda7421a67 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:21.317710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1e772b1d-bf93-47fd-89d9-b3bda7421a67 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:21.318194 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1e772b1d-bf93-47fd-89d9-b3bda7421a67 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:21.318627 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1e772b1d-bf93-47fd-89d9-b3bda7421a67 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:21.319322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1e772b1d-bf93-47fd-89d9-b3bda7421a67 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:21.319772 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1e772b1d-bf93-47fd-89d9-b3bda7421a67 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:21.320231 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1e772b1d-bf93-47fd-89d9-b3bda7421a67 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:21.541256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-606e21c1-f47c-4a5a-8623-1298fe6bb947 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:21.550041 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-606e21c1-f47c-4a5a-8623-1298fe6bb947 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:21.552612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-606e21c1-f47c-4a5a-8623-1298fe6bb947 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:21.553824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-606e21c1-f47c-4a5a-8623-1298fe6bb947 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1720 microversion: 2.20 time: 0.377641 Aug 21 01:31:21.554549 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 390/764] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:21 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1720 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:21.586450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1e772b1d-bf93-47fd-89d9-b3bda7421a67 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:21.596155 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1e772b1d-bf93-47fd-89d9-b3bda7421a67 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1366 microversion: 2.1 time: 0.289704 Aug 21 01:31:21.596855 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 375/765] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:21 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1366 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:22.579290 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2acfff5c-89c8-4baf-bf6b-c2f5f8ebfd93 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:22.580640 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-2acfff5c-89c8-4baf-bf6b-c2f5f8ebfd93 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:22.587847 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2acfff5c-89c8-4baf-bf6b-c2f5f8ebfd93 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:22.588417 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2acfff5c-89c8-4baf-bf6b-c2f5f8ebfd93 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:22.588905 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2acfff5c-89c8-4baf-bf6b-c2f5f8ebfd93 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:22.589740 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2acfff5c-89c8-4baf-bf6b-c2f5f8ebfd93 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:22.590754 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2acfff5c-89c8-4baf-bf6b-c2f5f8ebfd93 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:22.591186 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2acfff5c-89c8-4baf-bf6b-c2f5f8ebfd93 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:22.613385 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:31:22.619547 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9c2fe5aa-1ee2-43db-b90d-b9622f2d3031 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:22.620958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9c2fe5aa-1ee2-43db-b90d-b9622f2d3031 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:22.647664 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c2fe5aa-1ee2-43db-b90d-b9622f2d3031 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:22.647664 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c2fe5aa-1ee2-43db-b90d-b9622f2d3031 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:22.647664 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c2fe5aa-1ee2-43db-b90d-b9622f2d3031 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:22.647664 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c2fe5aa-1ee2-43db-b90d-b9622f2d3031 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:22.648056 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c2fe5aa-1ee2-43db-b90d-b9622f2d3031 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:22.648056 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c2fe5aa-1ee2-43db-b90d-b9622f2d3031 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:22.804287 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2acfff5c-89c8-4baf-bf6b-c2f5f8ebfd93 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:22.814345 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2acfff5c-89c8-4baf-bf6b-c2f5f8ebfd93 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:22.817142 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2acfff5c-89c8-4baf-bf6b-c2f5f8ebfd93 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:22.818475 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2acfff5c-89c8-4baf-bf6b-c2f5f8ebfd93 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1720 microversion: 2.20 time: 0.243889 Aug 21 01:31:22.819041 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 391/766] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:22 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1720 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:22.838384 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9c2fe5aa-1ee2-43db-b90d-b9622f2d3031 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:22.847685 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9c2fe5aa-1ee2-43db-b90d-b9622f2d3031 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1366 microversion: 2.1 time: 0.231894 Aug 21 01:31:22.848281 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 376/767] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:22 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1366 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:23.481537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-59bd6b3b-e2c5-4a52-9443-0f81acf1812e req-11f7c031-2aab-4a0f-9e3f-3439575080a4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1a9e859-eddf-4f45-8e02-168201b29a63", "name": "network-vif-plugged", "status": "completed", "tag": "316e8409-378a-47c3-a927-8b6aea8ed8bd"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:23.491686 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-59bd6b3b-e2c5-4a52-9443-0f81acf1812e req-11f7c031-2aab-4a0f-9e3f-3439575080a4 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:23.492056 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-59bd6b3b-e2c5-4a52-9443-0f81acf1812e req-11f7c031-2aab-4a0f-9e3f-3439575080a4 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:23.492583 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-59bd6b3b-e2c5-4a52-9443-0f81acf1812e req-11f7c031-2aab-4a0f-9e3f-3439575080a4 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:23.505117 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-59bd6b3b-e2c5-4a52-9443-0f81acf1812e req-11f7c031-2aab-4a0f-9e3f-3439575080a4 service nova] Creating event network-vif-plugged:316e8409-378a-47c3-a927-8b6aea8ed8bd for instance d1a9e859-eddf-4f45-8e02-168201b29a63 on hv-cinder-maste Aug 21 01:31:23.510328 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-59bd6b3b-e2c5-4a52-9443-0f81acf1812e req-11f7c031-2aab-4a0f-9e3f-3439575080a4 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097884 Aug 21 01:31:23.541024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 392/768] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:31:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:23.847377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b70e9d8f-6193-498a-9db4-57f6508e4350 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:23.847774 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-b70e9d8f-6193-498a-9db4-57f6508e4350 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:23.849886 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=6 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:31:23.859976 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b70e9d8f-6193-498a-9db4-57f6508e4350 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:23.860872 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b70e9d8f-6193-498a-9db4-57f6508e4350 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:23.861316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b70e9d8f-6193-498a-9db4-57f6508e4350 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:23.869920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9dd8fd94-bb6a-45d3-973e-235dcfc06ec5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:23.871346 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9dd8fd94-bb6a-45d3-973e-235dcfc06ec5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:23.877798 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b70e9d8f-6193-498a-9db4-57f6508e4350 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:23.878535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b70e9d8f-6193-498a-9db4-57f6508e4350 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:23.881161 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b70e9d8f-6193-498a-9db4-57f6508e4350 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:23.886114 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9dd8fd94-bb6a-45d3-973e-235dcfc06ec5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:23.889950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9dd8fd94-bb6a-45d3-973e-235dcfc06ec5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:23.890807 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9dd8fd94-bb6a-45d3-973e-235dcfc06ec5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:23.904810 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9dd8fd94-bb6a-45d3-973e-235dcfc06ec5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:23.905648 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9dd8fd94-bb6a-45d3-973e-235dcfc06ec5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:23.907087 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9dd8fd94-bb6a-45d3-973e-235dcfc06ec5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:24.009149 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=8 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:31:24.363389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b70e9d8f-6193-498a-9db4-57f6508e4350 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:24.375399 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b70e9d8f-6193-498a-9db4-57f6508e4350 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:24.377309 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-b70e9d8f-6193-498a-9db4-57f6508e4350 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:24.379276 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b70e9d8f-6193-498a-9db4-57f6508e4350 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1720 microversion: 2.20 time: 0.536312 Aug 21 01:31:24.380360 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 377/769] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:23 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1720 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:24.401981 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9dd8fd94-bb6a-45d3-973e-235dcfc06ec5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:24.421403 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9dd8fd94-bb6a-45d3-973e-235dcfc06ec5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1366 microversion: 2.1 time: 0.554075 Aug 21 01:31:24.423313 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 393/770] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:23 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1366 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:25.404060 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bf3385ae-3c72-4d78-983a-b6cd9dbac659 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:25.404728 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bf3385ae-3c72-4d78-983a-b6cd9dbac659 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:25.411839 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bf3385ae-3c72-4d78-983a-b6cd9dbac659 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:25.413794 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bf3385ae-3c72-4d78-983a-b6cd9dbac659 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:25.413794 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bf3385ae-3c72-4d78-983a-b6cd9dbac659 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:25.413794 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bf3385ae-3c72-4d78-983a-b6cd9dbac659 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:25.414287 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bf3385ae-3c72-4d78-983a-b6cd9dbac659 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:25.414810 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bf3385ae-3c72-4d78-983a-b6cd9dbac659 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:25.444720 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a4b9f930-48b7-4961-9be8-9eaabeb0a815 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:25.445277 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a4b9f930-48b7-4961-9be8-9eaabeb0a815 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:25.453879 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a4b9f930-48b7-4961-9be8-9eaabeb0a815 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:25.454370 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a4b9f930-48b7-4961-9be8-9eaabeb0a815 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:25.454828 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a4b9f930-48b7-4961-9be8-9eaabeb0a815 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:25.455533 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a4b9f930-48b7-4961-9be8-9eaabeb0a815 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:25.456186 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a4b9f930-48b7-4961-9be8-9eaabeb0a815 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:25.456613 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a4b9f930-48b7-4961-9be8-9eaabeb0a815 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:25.655113 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-bf3385ae-3c72-4d78-983a-b6cd9dbac659 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:25.663632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-bf3385ae-3c72-4d78-983a-b6cd9dbac659 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:25.665161 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-bf3385ae-3c72-4d78-983a-b6cd9dbac659 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:25.666377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bf3385ae-3c72-4d78-983a-b6cd9dbac659 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1720 microversion: 2.20 time: 0.267497 Aug 21 01:31:25.670094 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 378/771] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:25 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1720 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:25.671180 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a4b9f930-48b7-4961-9be8-9eaabeb0a815 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:25.680696 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a4b9f930-48b7-4961-9be8-9eaabeb0a815 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1366 microversion: 2.1 time: 0.240356 Aug 21 01:31:25.681270 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 394/772] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:25 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1366 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:26.691370 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-85c39371-d8f9-453e-b8dd-336d6a26bea2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:26.691983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-85c39371-d8f9-453e-b8dd-336d6a26bea2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:26.699545 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-85c39371-d8f9-453e-b8dd-336d6a26bea2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:26.700048 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-85c39371-d8f9-453e-b8dd-336d6a26bea2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:26.700774 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-85c39371-d8f9-453e-b8dd-336d6a26bea2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:26.702274 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-85c39371-d8f9-453e-b8dd-336d6a26bea2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:26.702910 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-85c39371-d8f9-453e-b8dd-336d6a26bea2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:26.703424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-85c39371-d8f9-453e-b8dd-336d6a26bea2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:26.704377 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7f34420c-79c3-477a-86dd-2aa62181793d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:26.705070 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7f34420c-79c3-477a-86dd-2aa62181793d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:26.713304 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f34420c-79c3-477a-86dd-2aa62181793d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:26.714080 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f34420c-79c3-477a-86dd-2aa62181793d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:26.714610 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f34420c-79c3-477a-86dd-2aa62181793d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:26.715387 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f34420c-79c3-477a-86dd-2aa62181793d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:26.715963 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f34420c-79c3-477a-86dd-2aa62181793d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:26.716655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f34420c-79c3-477a-86dd-2aa62181793d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:26.908005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-85c39371-d8f9-453e-b8dd-336d6a26bea2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:26.910538 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7f34420c-79c3-477a-86dd-2aa62181793d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:26.918683 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7f34420c-79c3-477a-86dd-2aa62181793d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1555 microversion: 2.1 time: 0.219790 Aug 21 01:31:26.919237 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-85c39371-d8f9-453e-b8dd-336d6a26bea2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:26.922041 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-85c39371-d8f9-453e-b8dd-336d6a26bea2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:26.923609 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-85c39371-d8f9-453e-b8dd-336d6a26bea2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1720 microversion: 2.20 time: 0.236179 Aug 21 01:31:26.924549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 379/773] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:26 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1720 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:26.936572 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 395/774] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:26 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1555 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:26.952127 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8b6951ca-5b3c-49db-9e23-f91cdec5a7d6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:26.952895 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8b6951ca-5b3c-49db-9e23-f91cdec5a7d6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:26.959053 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8b6951ca-5b3c-49db-9e23-f91cdec5a7d6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:26.959538 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8b6951ca-5b3c-49db-9e23-f91cdec5a7d6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:26.959977 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8b6951ca-5b3c-49db-9e23-f91cdec5a7d6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:26.960702 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8b6951ca-5b3c-49db-9e23-f91cdec5a7d6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:26.961162 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8b6951ca-5b3c-49db-9e23-f91cdec5a7d6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:26.961595 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8b6951ca-5b3c-49db-9e23-f91cdec5a7d6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:27.044089 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8b6951ca-5b3c-49db-9e23-f91cdec5a7d6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.096519 Aug 21 01:31:27.045770 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 380/775] 10.222.0.47 () {58 vars in 1275 bytes} [Sun Aug 21 01:31:26 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/os-interface => generated 284 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:27.958282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6178f4d9-ca6c-438f-90fc-59f2434fdad3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:27.958893 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6178f4d9-ca6c-438f-90fc-59f2434fdad3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:27.965624 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6178f4d9-ca6c-438f-90fc-59f2434fdad3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:27.966104 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6178f4d9-ca6c-438f-90fc-59f2434fdad3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:27.966537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6178f4d9-ca6c-438f-90fc-59f2434fdad3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:27.967862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6178f4d9-ca6c-438f-90fc-59f2434fdad3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:27.968338 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6178f4d9-ca6c-438f-90fc-59f2434fdad3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:27.968780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6178f4d9-ca6c-438f-90fc-59f2434fdad3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:28.122626 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-f064859c-034e-4907-8a87-3c93fe387e18 req-94451fde-ecab-445d-b373-d100601ec8b3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1a9e859-eddf-4f45-8e02-168201b29a63", "tag": "316e8409-378a-47c3-a927-8b6aea8ed8bd"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:28.133600 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-f064859c-034e-4907-8a87-3c93fe387e18 req-94451fde-ecab-445d-b373-d100601ec8b3 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:28.134574 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-f064859c-034e-4907-8a87-3c93fe387e18 req-94451fde-ecab-445d-b373-d100601ec8b3 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:28.135014 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-f064859c-034e-4907-8a87-3c93fe387e18 req-94451fde-ecab-445d-b373-d100601ec8b3 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:28.148507 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-f064859c-034e-4907-8a87-3c93fe387e18 req-94451fde-ecab-445d-b373-d100601ec8b3 service nova] Creating event network-changed:316e8409-378a-47c3-a927-8b6aea8ed8bd for instance d1a9e859-eddf-4f45-8e02-168201b29a63 on hv-cinder-maste Aug 21 01:31:28.149408 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6178f4d9-ca6c-438f-90fc-59f2434fdad3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:28.153787 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-f064859c-034e-4907-8a87-3c93fe387e18 req-94451fde-ecab-445d-b373-d100601ec8b3 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035377 Aug 21 01:31:28.154633 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 381/776] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:31:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:28.159919 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6178f4d9-ca6c-438f-90fc-59f2434fdad3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:28.161628 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6178f4d9-ca6c-438f-90fc-59f2434fdad3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:28.162862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6178f4d9-ca6c-438f-90fc-59f2434fdad3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1720 microversion: 2.20 time: 0.208646 Aug 21 01:31:28.168415 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 396/777] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:27 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1720 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:29.187255 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ac2c3d26-0f7e-4b6f-9ce1-1b93e76f04d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:29.187910 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ac2c3d26-0f7e-4b6f-9ce1-1b93e76f04d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:29.194538 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ac2c3d26-0f7e-4b6f-9ce1-1b93e76f04d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:29.196361 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ac2c3d26-0f7e-4b6f-9ce1-1b93e76f04d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:29.196361 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ac2c3d26-0f7e-4b6f-9ce1-1b93e76f04d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:29.196775 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ac2c3d26-0f7e-4b6f-9ce1-1b93e76f04d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:29.197244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ac2c3d26-0f7e-4b6f-9ce1-1b93e76f04d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:29.197677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ac2c3d26-0f7e-4b6f-9ce1-1b93e76f04d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:29.343687 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ac2c3d26-0f7e-4b6f-9ce1-1b93e76f04d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:29.355100 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ac2c3d26-0f7e-4b6f-9ce1-1b93e76f04d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:29.356784 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ac2c3d26-0f7e-4b6f-9ce1-1b93e76f04d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:29.358124 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ac2c3d26-0f7e-4b6f-9ce1-1b93e76f04d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1720 microversion: 2.20 time: 0.175741 Aug 21 01:31:29.358693 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 382/778] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:29 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1720 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:30.381112 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6e59241b-aa7d-423b-82c5-09141cc928fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:30.381718 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6e59241b-aa7d-423b-82c5-09141cc928fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:30.388340 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6e59241b-aa7d-423b-82c5-09141cc928fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:30.388826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6e59241b-aa7d-423b-82c5-09141cc928fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:30.389260 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6e59241b-aa7d-423b-82c5-09141cc928fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:30.389947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6e59241b-aa7d-423b-82c5-09141cc928fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:30.390398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6e59241b-aa7d-423b-82c5-09141cc928fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:30.390827 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6e59241b-aa7d-423b-82c5-09141cc928fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:30.525072 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6e59241b-aa7d-423b-82c5-09141cc928fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:30.534639 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6e59241b-aa7d-423b-82c5-09141cc928fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:30.536199 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6e59241b-aa7d-423b-82c5-09141cc928fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:30.537372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6e59241b-aa7d-423b-82c5-09141cc928fb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1741 microversion: 2.20 time: 0.160191 Aug 21 01:31:30.537936 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 397/779] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:30 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1741 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:31.560665 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6c6ddfb6-e97e-4961-b1f8-56862207f1eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:31.561267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6c6ddfb6-e97e-4961-b1f8-56862207f1eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:31.564209 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=1,nova_api:INSERT=1,nova_api:DELETE=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:31:31.568867 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c6ddfb6-e97e-4961-b1f8-56862207f1eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:31.569351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c6ddfb6-e97e-4961-b1f8-56862207f1eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:31.569839 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c6ddfb6-e97e-4961-b1f8-56862207f1eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:31.575083 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c6ddfb6-e97e-4961-b1f8-56862207f1eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:31.575083 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c6ddfb6-e97e-4961-b1f8-56862207f1eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:31.575083 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c6ddfb6-e97e-4961-b1f8-56862207f1eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:31.714085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6c6ddfb6-e97e-4961-b1f8-56862207f1eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:31.722641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6c6ddfb6-e97e-4961-b1f8-56862207f1eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:31.724294 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6c6ddfb6-e97e-4961-b1f8-56862207f1eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:31.725524 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6c6ddfb6-e97e-4961-b1f8-56862207f1eb tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1741 microversion: 2.20 time: 0.168778 Aug 21 01:31:31.726090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 383/780] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:31 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1741 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:32.748785 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ee5d70ef-aada-43c0-a410-b45da4126590 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:32.750080 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-ee5d70ef-aada-43c0-a410-b45da4126590 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:32.756844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ee5d70ef-aada-43c0-a410-b45da4126590 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:32.757584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ee5d70ef-aada-43c0-a410-b45da4126590 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:32.758281 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ee5d70ef-aada-43c0-a410-b45da4126590 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:32.759723 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ee5d70ef-aada-43c0-a410-b45da4126590 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:32.760557 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ee5d70ef-aada-43c0-a410-b45da4126590 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:32.761248 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ee5d70ef-aada-43c0-a410-b45da4126590 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:32.786135 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=100 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:31:32.922439 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ee5d70ef-aada-43c0-a410-b45da4126590 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:32.934552 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ee5d70ef-aada-43c0-a410-b45da4126590 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:32.936477 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ee5d70ef-aada-43c0-a410-b45da4126590 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:32.938242 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ee5d70ef-aada-43c0-a410-b45da4126590 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1736 microversion: 2.20 time: 0.192747 Aug 21 01:31:32.938810 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 398/781] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:32 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1736 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:33.353856 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-5f5e9da6-2bbb-4d07-ae8b-15227a32f7e9 req-0d82db4b-d0da-4c01-8430-60f32bc5d83e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a5b475aa-486a-4957-8b54-aa1ee7438a8a", "name": "network-vif-plugged", "status": "completed", "tag": "d3beeb80-5980-4de1-b832-2d4a1e1dae1c"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:33.366572 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-5f5e9da6-2bbb-4d07-ae8b-15227a32f7e9 req-0d82db4b-d0da-4c01-8430-60f32bc5d83e service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:33.367052 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-5f5e9da6-2bbb-4d07-ae8b-15227a32f7e9 req-0d82db4b-d0da-4c01-8430-60f32bc5d83e service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:33.367476 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-5f5e9da6-2bbb-4d07-ae8b-15227a32f7e9 req-0d82db4b-d0da-4c01-8430-60f32bc5d83e service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:33.379795 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-5f5e9da6-2bbb-4d07-ae8b-15227a32f7e9 req-0d82db4b-d0da-4c01-8430-60f32bc5d83e service nova] Creating event network-vif-plugged:d3beeb80-5980-4de1-b832-2d4a1e1dae1c for instance a5b475aa-486a-4957-8b54-aa1ee7438a8a on hv-cinder-maste Aug 21 01:31:33.385502 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-5f5e9da6-2bbb-4d07-ae8b-15227a32f7e9 req-0d82db4b-d0da-4c01-8430-60f32bc5d83e service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035388 Aug 21 01:31:33.386092 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 384/782] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:31:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:33.491902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-bb7fe79f-7538-4a4e-acc9-420b58a43240 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:33.495315 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-bb7fe79f-7538-4a4e-acc9-420b58a43240 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:33.502673 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bb7fe79f-7538-4a4e-acc9-420b58a43240 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:33.503161 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bb7fe79f-7538-4a4e-acc9-420b58a43240 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:33.503597 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bb7fe79f-7538-4a4e-acc9-420b58a43240 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:33.504322 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bb7fe79f-7538-4a4e-acc9-420b58a43240 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:33.504780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bb7fe79f-7538-4a4e-acc9-420b58a43240 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:33.505258 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bb7fe79f-7538-4a4e-acc9-420b58a43240 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:33.533434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-bb7fe79f-7538-4a4e-acc9-420b58a43240 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/ips" status: 200 len: 146 microversion: 2.1 time: 0.051099 Aug 21 01:31:33.534665 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 399/783] 10.222.0.47 () {58 vars in 1248 bytes} [Sun Aug 21 01:31:33 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/ips => generated 146 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:33.965578 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-5d86b963-653d-491f-af16-6cba46d72ee8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:33.968544 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-5d86b963-653d-491f-af16-6cba46d72ee8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:33.975690 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d86b963-653d-491f-af16-6cba46d72ee8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:33.976427 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d86b963-653d-491f-af16-6cba46d72ee8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:33.977075 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d86b963-653d-491f-af16-6cba46d72ee8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:33.977942 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d86b963-653d-491f-af16-6cba46d72ee8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:33.978563 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d86b963-653d-491f-af16-6cba46d72ee8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:33.979046 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d86b963-653d-491f-af16-6cba46d72ee8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:34.145494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5d86b963-653d-491f-af16-6cba46d72ee8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:34.155355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5d86b963-653d-491f-af16-6cba46d72ee8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:34.157462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5d86b963-653d-491f-af16-6cba46d72ee8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:34.158635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5d86b963-653d-491f-af16-6cba46d72ee8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1736 microversion: 2.20 time: 0.199529 Aug 21 01:31:34.159538 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 385/784] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:33 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1736 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:35.182809 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-58c997b1-af57-424f-b738-beca3a911754 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:35.182809 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-58c997b1-af57-424f-b738-beca3a911754 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:35.188755 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-58c997b1-af57-424f-b738-beca3a911754 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:35.189238 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-58c997b1-af57-424f-b738-beca3a911754 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:35.189670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-58c997b1-af57-424f-b738-beca3a911754 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:35.190349 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-58c997b1-af57-424f-b738-beca3a911754 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:35.190799 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-58c997b1-af57-424f-b738-beca3a911754 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:35.191220 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-58c997b1-af57-424f-b738-beca3a911754 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:35.327037 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-58c997b1-af57-424f-b738-beca3a911754 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:35.336146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-58c997b1-af57-424f-b738-beca3a911754 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:35.337725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-58c997b1-af57-424f-b738-beca3a911754 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:35.338901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-58c997b1-af57-424f-b738-beca3a911754 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1736 microversion: 2.20 time: 0.161452 Aug 21 01:31:35.343173 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 400/785] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:35 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1736 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:35.381026 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ebef8418-cc43-4583-b01a-dc5159f4be4e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "30c9c194-809c-4a6e-b408-a7b8d79a3d5c"}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:35.382209 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ebef8418-cc43-4583-b01a-dc5159f4be4e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:35.388345 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ebef8418-cc43-4583-b01a-dc5159f4be4e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:35.388840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ebef8418-cc43-4583-b01a-dc5159f4be4e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:35.389281 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ebef8418-cc43-4583-b01a-dc5159f4be4e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:35.390055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ebef8418-cc43-4583-b01a-dc5159f4be4e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:35.390516 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ebef8418-cc43-4583-b01a-dc5159f4be4e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:35.391522 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ebef8418-cc43-4583-b01a-dc5159f4be4e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:36.346569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ebef8418-cc43-4583-b01a-dc5159f4be4e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "POST /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.972405 Aug 21 01:31:36.347847 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 386/786] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:31:35 2022] POST /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments => generated 194 bytes in 975 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:36.369524 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4930b868-8d7a-4583-a802-5c9d95015a0d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:36.370122 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4930b868-8d7a-4583-a802-5c9d95015a0d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:36.389357 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4930b868-8d7a-4583-a802-5c9d95015a0d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:36.390118 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4930b868-8d7a-4583-a802-5c9d95015a0d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:36.390704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4930b868-8d7a-4583-a802-5c9d95015a0d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:36.392452 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4930b868-8d7a-4583-a802-5c9d95015a0d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:36.393078 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4930b868-8d7a-4583-a802-5c9d95015a0d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:36.393668 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4930b868-8d7a-4583-a802-5c9d95015a0d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:36.576984 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4930b868-8d7a-4583-a802-5c9d95015a0d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:36.586414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4930b868-8d7a-4583-a802-5c9d95015a0d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:36.588211 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4930b868-8d7a-4583-a802-5c9d95015a0d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:36.589396 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4930b868-8d7a-4583-a802-5c9d95015a0d tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1736 microversion: 2.20 time: 0.226164 Aug 21 01:31:36.589943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 401/787] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:36 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1736 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:37.616766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2045c4d6-be2f-4f55-b6d7-b9dbcf63c25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:37.617384 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2045c4d6-be2f-4f55-b6d7-b9dbcf63c25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:37.627511 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2045c4d6-be2f-4f55-b6d7-b9dbcf63c25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:37.628020 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2045c4d6-be2f-4f55-b6d7-b9dbcf63c25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:37.628491 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2045c4d6-be2f-4f55-b6d7-b9dbcf63c25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:37.629212 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2045c4d6-be2f-4f55-b6d7-b9dbcf63c25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:37.629665 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2045c4d6-be2f-4f55-b6d7-b9dbcf63c25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:37.630093 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2045c4d6-be2f-4f55-b6d7-b9dbcf63c25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:37.775105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2045c4d6-be2f-4f55-b6d7-b9dbcf63c25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:37.787259 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2045c4d6-be2f-4f55-b6d7-b9dbcf63c25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:37.789155 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2045c4d6-be2f-4f55-b6d7-b9dbcf63c25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:37.790630 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2045c4d6-be2f-4f55-b6d7-b9dbcf63c25f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1736 microversion: 2.20 time: 0.179101 Aug 21 01:31:37.804317 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 387/788] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:37 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1736 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:38.815800 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ef79998d-37c4-4d7f-8629-d56141380f65 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:38.816841 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-ef79998d-37c4-4d7f-8629-d56141380f65 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:38.823661 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ef79998d-37c4-4d7f-8629-d56141380f65 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:38.824595 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ef79998d-37c4-4d7f-8629-d56141380f65 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:38.825275 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ef79998d-37c4-4d7f-8629-d56141380f65 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:38.826457 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ef79998d-37c4-4d7f-8629-d56141380f65 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:38.827084 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ef79998d-37c4-4d7f-8629-d56141380f65 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:38.827673 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ef79998d-37c4-4d7f-8629-d56141380f65 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:38.987332 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ef79998d-37c4-4d7f-8629-d56141380f65 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:38.998244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ef79998d-37c4-4d7f-8629-d56141380f65 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:38.999794 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ef79998d-37c4-4d7f-8629-d56141380f65 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:39.001356 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ef79998d-37c4-4d7f-8629-d56141380f65 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1736 microversion: 2.20 time: 0.190116 Aug 21 01:31:39.002023 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 402/789] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:38 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1736 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:40.025071 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a4af4ef6-c69f-4256-9368-5a34ec6ba46b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:40.025785 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a4af4ef6-c69f-4256-9368-5a34ec6ba46b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:40.037502 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4af4ef6-c69f-4256-9368-5a34ec6ba46b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:40.037502 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4af4ef6-c69f-4256-9368-5a34ec6ba46b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:40.037842 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4af4ef6-c69f-4256-9368-5a34ec6ba46b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:40.038276 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4af4ef6-c69f-4256-9368-5a34ec6ba46b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:40.038735 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4af4ef6-c69f-4256-9368-5a34ec6ba46b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:40.039173 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a4af4ef6-c69f-4256-9368-5a34ec6ba46b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:40.061418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:31:40.202926 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a4af4ef6-c69f-4256-9368-5a34ec6ba46b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:40.213408 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a4af4ef6-c69f-4256-9368-5a34ec6ba46b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:40.214944 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a4af4ef6-c69f-4256-9368-5a34ec6ba46b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:40.216231 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a4af4ef6-c69f-4256-9368-5a34ec6ba46b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1736 microversion: 2.20 time: 0.195060 Aug 21 01:31:40.231440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 388/790] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:40 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1736 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:41.239667 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-cd9ffab5-99bc-46fb-83fb-eda03dc88db6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:41.240331 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-cd9ffab5-99bc-46fb-83fb-eda03dc88db6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:41.246718 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cd9ffab5-99bc-46fb-83fb-eda03dc88db6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:41.247201 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cd9ffab5-99bc-46fb-83fb-eda03dc88db6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:41.247636 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cd9ffab5-99bc-46fb-83fb-eda03dc88db6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:41.248354 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cd9ffab5-99bc-46fb-83fb-eda03dc88db6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:41.248804 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cd9ffab5-99bc-46fb-83fb-eda03dc88db6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:41.249228 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cd9ffab5-99bc-46fb-83fb-eda03dc88db6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:41.391241 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-cd9ffab5-99bc-46fb-83fb-eda03dc88db6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:41.399497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-cd9ffab5-99bc-46fb-83fb-eda03dc88db6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:41.401049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-cd9ffab5-99bc-46fb-83fb-eda03dc88db6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:41.402204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-cd9ffab5-99bc-46fb-83fb-eda03dc88db6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1736 microversion: 2.20 time: 0.167192 Aug 21 01:31:41.402775 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 403/791] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:41 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1736 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:41.976996 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-773f1ac0-8f2f-437e-a40f-b0f2d50a5982 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:41.977589 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-773f1ac0-8f2f-437e-a40f-b0f2d50a5982 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:41.984336 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-773f1ac0-8f2f-437e-a40f-b0f2d50a5982 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:41.984576 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-773f1ac0-8f2f-437e-a40f-b0f2d50a5982 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:41.985005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-773f1ac0-8f2f-437e-a40f-b0f2d50a5982 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:41.985710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-773f1ac0-8f2f-437e-a40f-b0f2d50a5982 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:41.986145 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-773f1ac0-8f2f-437e-a40f-b0f2d50a5982 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:41.986565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-773f1ac0-8f2f-437e-a40f-b0f2d50a5982 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:42.009556 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-773f1ac0-8f2f-437e-a40f-b0f2d50a5982 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lazy-loading 'flavor' on Instance uuid 119204ec-e0f8-439e-b985-b934d251684f {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:31:42.439223 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6edb6b32-c200-46c8-a32e-4438c8519f90 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:42.439832 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6edb6b32-c200-46c8-a32e-4438c8519f90 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:42.447087 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6edb6b32-c200-46c8-a32e-4438c8519f90 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:42.447863 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6edb6b32-c200-46c8-a32e-4438c8519f90 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:42.448494 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6edb6b32-c200-46c8-a32e-4438c8519f90 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:42.448911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-773f1ac0-8f2f-437e-a40f-b0f2d50a5982 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "POST /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f/action" status: 202 len: 0 microversion: 2.1 time: 0.475835 Aug 21 01:31:42.449570 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6edb6b32-c200-46c8-a32e-4438c8519f90 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:42.450195 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6edb6b32-c200-46c8-a32e-4438c8519f90 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:42.450856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6edb6b32-c200-46c8-a32e-4438c8519f90 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:42.455652 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 389/792] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:31:41 2022] POST /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f/action => generated 0 bytes in 483 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:31:42.462198 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-0a3fbbbc-b70f-47c8-9a1a-b7dad51daddb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:42.463093 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-0a3fbbbc-b70f-47c8-9a1a-b7dad51daddb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:42.470731 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0a3fbbbc-b70f-47c8-9a1a-b7dad51daddb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:42.471235 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0a3fbbbc-b70f-47c8-9a1a-b7dad51daddb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:42.471677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0a3fbbbc-b70f-47c8-9a1a-b7dad51daddb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:42.472669 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0a3fbbbc-b70f-47c8-9a1a-b7dad51daddb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:42.473132 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0a3fbbbc-b70f-47c8-9a1a-b7dad51daddb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:42.473566 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0a3fbbbc-b70f-47c8-9a1a-b7dad51daddb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:42.687674 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6edb6b32-c200-46c8-a32e-4438c8519f90 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:42.691945 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0a3fbbbc-b70f-47c8-9a1a-b7dad51daddb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:42.697789 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6edb6b32-c200-46c8-a32e-4438c8519f90 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:42.698535 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6edb6b32-c200-46c8-a32e-4438c8519f90 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:42.700298 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6edb6b32-c200-46c8-a32e-4438c8519f90 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1736 microversion: 2.20 time: 0.269169 Aug 21 01:31:42.700568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 404/793] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:42 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1736 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:31:42.700694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-0a3fbbbc-b70f-47c8-9a1a-b7dad51daddb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.243214 Aug 21 01:31:42.701160 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 390/794] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:42 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:43.722596 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-785c38f1-4830-4f8e-ab9b-51f2ce48e70f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:43.723291 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-785c38f1-4830-4f8e-ab9b-51f2ce48e70f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:43.723939 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8b59272f-cdc6-43ac-a20c-7d09b347bfec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:43.724590 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8b59272f-cdc6-43ac-a20c-7d09b347bfec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:43.729603 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-785c38f1-4830-4f8e-ab9b-51f2ce48e70f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:43.730129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-785c38f1-4830-4f8e-ab9b-51f2ce48e70f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:43.730625 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-785c38f1-4830-4f8e-ab9b-51f2ce48e70f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:43.731022 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8b59272f-cdc6-43ac-a20c-7d09b347bfec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:43.731329 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-785c38f1-4830-4f8e-ab9b-51f2ce48e70f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:43.731508 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8b59272f-cdc6-43ac-a20c-7d09b347bfec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:43.731783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-785c38f1-4830-4f8e-ab9b-51f2ce48e70f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:43.731953 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8b59272f-cdc6-43ac-a20c-7d09b347bfec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:43.732408 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-785c38f1-4830-4f8e-ab9b-51f2ce48e70f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:43.732675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8b59272f-cdc6-43ac-a20c-7d09b347bfec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:43.733130 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8b59272f-cdc6-43ac-a20c-7d09b347bfec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:43.733559 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8b59272f-cdc6-43ac-a20c-7d09b347bfec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:43.939237 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-785c38f1-4830-4f8e-ab9b-51f2ce48e70f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:43.947910 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8b59272f-cdc6-43ac-a20c-7d09b347bfec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:43.951187 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-785c38f1-4830-4f8e-ab9b-51f2ce48e70f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.232988 Aug 21 01:31:43.953317 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 405/795] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:43 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:43.958153 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8b59272f-cdc6-43ac-a20c-7d09b347bfec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:43.960042 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8b59272f-cdc6-43ac-a20c-7d09b347bfec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:43.961310 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8b59272f-cdc6-43ac-a20c-7d09b347bfec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1736 microversion: 2.20 time: 0.241207 Aug 21 01:31:43.964155 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 391/796] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:43 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1736 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:31:44.974653 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-185aec5e-151d-4c7f-938c-76057e0d340e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:44.975450 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-185aec5e-151d-4c7f-938c-76057e0d340e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:44.982574 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-185aec5e-151d-4c7f-938c-76057e0d340e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:44.983112 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-185aec5e-151d-4c7f-938c-76057e0d340e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:44.983694 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-185aec5e-151d-4c7f-938c-76057e0d340e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:44.984294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-185aec5e-151d-4c7f-938c-76057e0d340e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:44.984803 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-185aec5e-151d-4c7f-938c-76057e0d340e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:44.985741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-185aec5e-151d-4c7f-938c-76057e0d340e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:44.990159 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d6f74d31-040c-4b4f-8753-144faa426887 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:44.990159 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d6f74d31-040c-4b4f-8753-144faa426887 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:44.996549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6f74d31-040c-4b4f-8753-144faa426887 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:44.997065 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6f74d31-040c-4b4f-8753-144faa426887 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:44.997586 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6f74d31-040c-4b4f-8753-144faa426887 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:44.998325 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6f74d31-040c-4b4f-8753-144faa426887 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:44.998785 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6f74d31-040c-4b4f-8753-144faa426887 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:44.999210 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6f74d31-040c-4b4f-8753-144faa426887 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:45.197750 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-185aec5e-151d-4c7f-938c-76057e0d340e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:45.200319 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d6f74d31-040c-4b4f-8753-144faa426887 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:45.206390 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-185aec5e-151d-4c7f-938c-76057e0d340e tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.235844 Aug 21 01:31:45.206962 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 406/797] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:44 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:45.208946 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d6f74d31-040c-4b4f-8753-144faa426887 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:45.210590 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d6f74d31-040c-4b4f-8753-144faa426887 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:45.211808 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d6f74d31-040c-4b4f-8753-144faa426887 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1718 microversion: 2.20 time: 0.226963 Aug 21 01:31:45.212468 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-622ecfc3-90e2-42ae-9eb0-fbaa55242aab tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:45.214236 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-622ecfc3-90e2-42ae-9eb0-fbaa55242aab tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:45.220522 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 392/798] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:44 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1718 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:31:45.221506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-622ecfc3-90e2-42ae-9eb0-fbaa55242aab tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:45.221721 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-622ecfc3-90e2-42ae-9eb0-fbaa55242aab tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:45.222194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-622ecfc3-90e2-42ae-9eb0-fbaa55242aab tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:45.222981 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-622ecfc3-90e2-42ae-9eb0-fbaa55242aab tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:45.223503 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-622ecfc3-90e2-42ae-9eb0-fbaa55242aab tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:45.224008 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-622ecfc3-90e2-42ae-9eb0-fbaa55242aab tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:45.257512 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-622ecfc3-90e2-42ae-9eb0-fbaa55242aab tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049514 Aug 21 01:31:45.258083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 407/799] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:31:45 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:45.268869 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-753122b3-6536-499d-834b-05ebd62cfa23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:45.269361 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-753122b3-6536-499d-834b-05ebd62cfa23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:45.275538 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-753122b3-6536-499d-834b-05ebd62cfa23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:45.276025 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-753122b3-6536-499d-834b-05ebd62cfa23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:45.276493 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-753122b3-6536-499d-834b-05ebd62cfa23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:45.277252 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-753122b3-6536-499d-834b-05ebd62cfa23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:45.277705 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-753122b3-6536-499d-834b-05ebd62cfa23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:45.278130 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-753122b3-6536-499d-834b-05ebd62cfa23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:45.308040 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-753122b3-6536-499d-834b-05ebd62cfa23 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments/30c9c194-809c-4a6e-b408-a7b8d79a3d5c" status: 200 len: 194 microversion: 2.1 time: 0.043173 Aug 21 01:31:45.308670 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 393/800] 10.222.0.47 () {58 vars in 1413 bytes} [Sun Aug 21 01:31:45 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments/30c9c194-809c-4a6e-b408-a7b8d79a3d5c => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:46.229142 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d57a4d8a-f9a9-460b-86db-1e5e01ff4eb2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:46.230014 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d57a4d8a-f9a9-460b-86db-1e5e01ff4eb2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:46.236803 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-55b23670-a364-4213-8a4e-5717a085e7f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:46.237387 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d57a4d8a-f9a9-460b-86db-1e5e01ff4eb2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:46.237387 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d57a4d8a-f9a9-460b-86db-1e5e01ff4eb2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:46.237614 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-55b23670-a364-4213-8a4e-5717a085e7f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:46.237838 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d57a4d8a-f9a9-460b-86db-1e5e01ff4eb2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:46.238577 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d57a4d8a-f9a9-460b-86db-1e5e01ff4eb2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:46.239029 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d57a4d8a-f9a9-460b-86db-1e5e01ff4eb2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:46.239455 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d57a4d8a-f9a9-460b-86db-1e5e01ff4eb2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:46.245280 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55b23670-a364-4213-8a4e-5717a085e7f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:46.246025 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55b23670-a364-4213-8a4e-5717a085e7f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:46.246867 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55b23670-a364-4213-8a4e-5717a085e7f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:46.247667 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55b23670-a364-4213-8a4e-5717a085e7f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:46.248324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55b23670-a364-4213-8a4e-5717a085e7f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:46.248853 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55b23670-a364-4213-8a4e-5717a085e7f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:46.447932 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-55b23670-a364-4213-8a4e-5717a085e7f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:46.457750 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d57a4d8a-f9a9-460b-86db-1e5e01ff4eb2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:46.461275 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-55b23670-a364-4213-8a4e-5717a085e7f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:46.463916 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-55b23670-a364-4213-8a4e-5717a085e7f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:46.465667 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d57a4d8a-f9a9-460b-86db-1e5e01ff4eb2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.240438 Aug 21 01:31:46.465926 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-55b23670-a364-4213-8a4e-5717a085e7f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1718 microversion: 2.20 time: 0.234162 Aug 21 01:31:46.466231 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 408/801] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:46 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:46.466902 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 394/802] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:46 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1718 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:31:46.995459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8a94ea16-3fca-4cc5-b03d-a46db13aded4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ccf43391-fddc-43c3-b309-1561379e4aa1"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:46.997914 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8a94ea16-3fca-4cc5-b03d-a46db13aded4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:47.005798 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8a94ea16-3fca-4cc5-b03d-a46db13aded4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:47.006371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8a94ea16-3fca-4cc5-b03d-a46db13aded4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:47.006826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8a94ea16-3fca-4cc5-b03d-a46db13aded4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:47.007559 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8a94ea16-3fca-4cc5-b03d-a46db13aded4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:47.008033 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8a94ea16-3fca-4cc5-b03d-a46db13aded4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:47.008506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8a94ea16-3fca-4cc5-b03d-a46db13aded4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:47.301178 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-371b075b-b24a-4861-aadd-fb790b255007 req-5e8014f1-7693-4907-8385-66e91426ffbd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a5b475aa-486a-4957-8b54-aa1ee7438a8a", "name": "network-vif-unplugged", "status": "completed", "tag": "d3beeb80-5980-4de1-b832-2d4a1e1dae1c"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:47.311393 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-371b075b-b24a-4861-aadd-fb790b255007 req-5e8014f1-7693-4907-8385-66e91426ffbd service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:47.311868 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-371b075b-b24a-4861-aadd-fb790b255007 req-5e8014f1-7693-4907-8385-66e91426ffbd service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:47.312332 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-371b075b-b24a-4861-aadd-fb790b255007 req-5e8014f1-7693-4907-8385-66e91426ffbd service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:47.324147 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-371b075b-b24a-4861-aadd-fb790b255007 req-5e8014f1-7693-4907-8385-66e91426ffbd service nova] Creating event network-vif-unplugged:d3beeb80-5980-4de1-b832-2d4a1e1dae1c for instance a5b475aa-486a-4957-8b54-aa1ee7438a8a on hv-cinder-maste Aug 21 01:31:47.329835 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-371b075b-b24a-4861-aadd-fb790b255007 req-5e8014f1-7693-4907-8385-66e91426ffbd service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032334 Aug 21 01:31:47.338332 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 395/803] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:31:47 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:31:47.492653 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f18ea1e2-464b-4ae8-b376-69f3211eece3 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:47.493353 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f18ea1e2-464b-4ae8-b376-69f3211eece3 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:47.500929 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f18ea1e2-464b-4ae8-b376-69f3211eece3 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:47.501409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f18ea1e2-464b-4ae8-b376-69f3211eece3 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:47.501843 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f18ea1e2-464b-4ae8-b376-69f3211eece3 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:47.503142 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f18ea1e2-464b-4ae8-b376-69f3211eece3 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:47.503615 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f18ea1e2-464b-4ae8-b376-69f3211eece3 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:47.504041 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f18ea1e2-464b-4ae8-b376-69f3211eece3 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:47.648739 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f18ea1e2-464b-4ae8-b376-69f3211eece3 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:47.657103 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f18ea1e2-464b-4ae8-b376-69f3211eece3 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.169954 Aug 21 01:31:47.657742 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 396/804] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:47 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:47.662430 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-0f1a7b27-174a-4726-b6ab-58be206ee443 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:47.663482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-0f1a7b27-174a-4726-b6ab-58be206ee443 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:47.669825 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0f1a7b27-174a-4726-b6ab-58be206ee443 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:47.670409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0f1a7b27-174a-4726-b6ab-58be206ee443 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:47.674957 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0f1a7b27-174a-4726-b6ab-58be206ee443 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:47.674957 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0f1a7b27-174a-4726-b6ab-58be206ee443 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:47.674957 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0f1a7b27-174a-4726-b6ab-58be206ee443 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:47.674957 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0f1a7b27-174a-4726-b6ab-58be206ee443 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:47.820161 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0f1a7b27-174a-4726-b6ab-58be206ee443 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:47.829288 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0f1a7b27-174a-4726-b6ab-58be206ee443 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:47.830833 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0f1a7b27-174a-4726-b6ab-58be206ee443 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:47.831985 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-0f1a7b27-174a-4726-b6ab-58be206ee443 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1721 microversion: 2.20 time: 0.173313 Aug 21 01:31:47.832567 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 397/805] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:47 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1721 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:47.858007 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-91aeed7a-8929-4d6c-932a-6b158dea8ee5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9e1ea8e7-ee01-4d99-aa9c-9aaea6ae94e1"}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:47.860367 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-91aeed7a-8929-4d6c-932a-6b158dea8ee5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:47.866850 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91aeed7a-8929-4d6c-932a-6b158dea8ee5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:47.867335 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91aeed7a-8929-4d6c-932a-6b158dea8ee5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:47.867773 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91aeed7a-8929-4d6c-932a-6b158dea8ee5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:47.868494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91aeed7a-8929-4d6c-932a-6b158dea8ee5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:47.868951 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91aeed7a-8929-4d6c-932a-6b158dea8ee5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:47.869376 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-91aeed7a-8929-4d6c-932a-6b158dea8ee5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:48.314249 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8a94ea16-3fca-4cc5-b03d-a46db13aded4 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "POST /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.322099 Aug 21 01:31:48.315094 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 409/806] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:31:46 2022] POST /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments => generated 194 bytes in 1324 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:48.679320 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9fa1a1e7-ce15-42aa-9255-08f7daa7f026 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:48.680016 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9fa1a1e7-ce15-42aa-9255-08f7daa7f026 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:48.686369 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9fa1a1e7-ce15-42aa-9255-08f7daa7f026 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:48.686848 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9fa1a1e7-ce15-42aa-9255-08f7daa7f026 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:48.687285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9fa1a1e7-ce15-42aa-9255-08f7daa7f026 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:48.687978 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9fa1a1e7-ce15-42aa-9255-08f7daa7f026 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:48.688463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9fa1a1e7-ce15-42aa-9255-08f7daa7f026 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:48.688887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9fa1a1e7-ce15-42aa-9255-08f7daa7f026 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:48.828146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9fa1a1e7-ce15-42aa-9255-08f7daa7f026 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:48.837477 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9fa1a1e7-ce15-42aa-9255-08f7daa7f026 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.162411 Aug 21 01:31:48.838192 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 410/807] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:48 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:49.304892 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-91aeed7a-8929-4d6c-932a-6b158dea8ee5 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "POST /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.450550 Aug 21 01:31:49.305571 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 398/808] 10.222.0.47 () {62 vars in 1364 bytes} [Sun Aug 21 01:31:47 2022] POST /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/os-volume_attachments => generated 188 bytes in 1452 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 21 01:31:49.450927 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-db920440-4e47-4d32-9985-3c5730a5fa09 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:49.452699 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-db920440-4e47-4d32-9985-3c5730a5fa09 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:49.460044 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-db920440-4e47-4d32-9985-3c5730a5fa09 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:49.460621 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-db920440-4e47-4d32-9985-3c5730a5fa09 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:49.461098 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-db920440-4e47-4d32-9985-3c5730a5fa09 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:49.462008 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-db920440-4e47-4d32-9985-3c5730a5fa09 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:49.463437 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-db920440-4e47-4d32-9985-3c5730a5fa09 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:49.463913 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-db920440-4e47-4d32-9985-3c5730a5fa09 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:49.859051 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-440a5745-f300-4f90-bfc4-c7b6c9ac2af2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:49.859724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-440a5745-f300-4f90-bfc4-c7b6c9ac2af2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:49.871737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-440a5745-f300-4f90-bfc4-c7b6c9ac2af2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:49.871737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-440a5745-f300-4f90-bfc4-c7b6c9ac2af2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:49.871737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-440a5745-f300-4f90-bfc4-c7b6c9ac2af2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:49.871737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-440a5745-f300-4f90-bfc4-c7b6c9ac2af2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:49.872770 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-440a5745-f300-4f90-bfc4-c7b6c9ac2af2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:49.872770 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-440a5745-f300-4f90-bfc4-c7b6c9ac2af2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:50.038290 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-440a5745-f300-4f90-bfc4-c7b6c9ac2af2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:50.057208 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-440a5745-f300-4f90-bfc4-c7b6c9ac2af2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.201996 Aug 21 01:31:50.057208 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 399/809] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:49 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:50.236379 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=5,nova_cell1:INSERT=3 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:31:50.310921 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-db920440-4e47-4d32-9985-3c5730a5fa09 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "DELETE /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/os-volume_attachments/9e1ea8e7-ee01-4d99-aa9c-9aaea6ae94e1" status: 202 len: 0 microversion: 2.20 time: 0.864496 Aug 21 01:31:50.311435 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 411/810] 10.222.0.47 () {60 vars in 1457 bytes} [Sun Aug 21 01:31:49 2022] DELETE /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/os-volume_attachments/9e1ea8e7-ee01-4d99-aa9c-9aaea6ae94e1 => generated 0 bytes in 866 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:31:50.375368 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-60e5ebd9-121b-4824-92c0-c1de06d4b3d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:50.375956 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-60e5ebd9-121b-4824-92c0-c1de06d4b3d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:50.383331 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-60e5ebd9-121b-4824-92c0-c1de06d4b3d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:50.383815 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-60e5ebd9-121b-4824-92c0-c1de06d4b3d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:50.384288 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-60e5ebd9-121b-4824-92c0-c1de06d4b3d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:50.384990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-60e5ebd9-121b-4824-92c0-c1de06d4b3d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:50.385439 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-60e5ebd9-121b-4824-92c0-c1de06d4b3d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:50.385854 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-60e5ebd9-121b-4824-92c0-c1de06d4b3d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:50.607653 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-60e5ebd9-121b-4824-92c0-c1de06d4b3d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=121276) unshelve /opt/stack/nova/nova/compute/api.py:4578}} Aug 21 01:31:50.755342 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=6,nova_cell1:INSERT=4 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:31:50.986292 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-60e5ebd9-121b-4824-92c0-c1de06d4b3d4 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "POST /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/action" status: 202 len: 0 microversion: 2.20 time: 0.614823 Aug 21 01:31:50.993473 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 400/811] 10.222.0.47 () {62 vars in 1319 bytes} [Sun Aug 21 01:31:50 2022] POST /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/action => generated 0 bytes in 616 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Aug 21 01:31:50.997081 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-999413ed-ffa2-4c2b-bef1-c5fbe3648f6f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:50.997809 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-999413ed-ffa2-4c2b-bef1-c5fbe3648f6f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:51.005725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-999413ed-ffa2-4c2b-bef1-c5fbe3648f6f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:51.006220 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-999413ed-ffa2-4c2b-bef1-c5fbe3648f6f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:51.006667 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-999413ed-ffa2-4c2b-bef1-c5fbe3648f6f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:51.007373 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-999413ed-ffa2-4c2b-bef1-c5fbe3648f6f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:51.007843 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-999413ed-ffa2-4c2b-bef1-c5fbe3648f6f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:51.008299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-999413ed-ffa2-4c2b-bef1-c5fbe3648f6f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:51.079854 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4238250f-c6de-4da1-8bac-477862c1b666 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:51.080577 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4238250f-c6de-4da1-8bac-477862c1b666 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:51.086675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4238250f-c6de-4da1-8bac-477862c1b666 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:51.087162 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4238250f-c6de-4da1-8bac-477862c1b666 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:51.087597 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4238250f-c6de-4da1-8bac-477862c1b666 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:51.088541 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4238250f-c6de-4da1-8bac-477862c1b666 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:51.088777 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4238250f-c6de-4da1-8bac-477862c1b666 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:51.089199 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4238250f-c6de-4da1-8bac-477862c1b666 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:51.205737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-999413ed-ffa2-4c2b-bef1-c5fbe3648f6f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:51.219576 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-999413ed-ffa2-4c2b-bef1-c5fbe3648f6f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:51.223139 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-999413ed-ffa2-4c2b-bef1-c5fbe3648f6f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:51.225313 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-999413ed-ffa2-4c2b-bef1-c5fbe3648f6f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1669 microversion: 2.20 time: 0.232509 Aug 21 01:31:51.226732 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 412/812] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:50 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1669 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:51.303609 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4238250f-c6de-4da1-8bac-477862c1b666 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:51.314935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4238250f-c6de-4da1-8bac-477862c1b666 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.239075 Aug 21 01:31:51.316284 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 401/813] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:51 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:52.254403 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-dcdc6616-5b57-42ee-afd6-e357595b71bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:52.255066 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-dcdc6616-5b57-42ee-afd6-e357595b71bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:52.259008 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=48 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:31:52.263251 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dcdc6616-5b57-42ee-afd6-e357595b71bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:52.263738 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dcdc6616-5b57-42ee-afd6-e357595b71bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:52.264493 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dcdc6616-5b57-42ee-afd6-e357595b71bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:52.264793 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dcdc6616-5b57-42ee-afd6-e357595b71bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:52.265449 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dcdc6616-5b57-42ee-afd6-e357595b71bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:52.265723 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dcdc6616-5b57-42ee-afd6-e357595b71bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:52.338755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-344a8472-2857-4462-ba04-64e11295c046 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:52.348911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-344a8472-2857-4462-ba04-64e11295c046 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:52.348911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-344a8472-2857-4462-ba04-64e11295c046 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:52.348911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-344a8472-2857-4462-ba04-64e11295c046 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:52.348911 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-344a8472-2857-4462-ba04-64e11295c046 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:52.351666 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-344a8472-2857-4462-ba04-64e11295c046 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:52.352163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-344a8472-2857-4462-ba04-64e11295c046 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:52.352607 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-344a8472-2857-4462-ba04-64e11295c046 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:52.758517 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-dcdc6616-5b57-42ee-afd6-e357595b71bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:52.774990 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-dcdc6616-5b57-42ee-afd6-e357595b71bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:52.778480 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-dcdc6616-5b57-42ee-afd6-e357595b71bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:52.780725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-dcdc6616-5b57-42ee-afd6-e357595b71bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1669 microversion: 2.20 time: 0.531383 Aug 21 01:31:52.781931 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 413/814] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:52 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1669 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:52.852140 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-344a8472-2857-4462-ba04-64e11295c046 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:52.865655 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-344a8472-2857-4462-ba04-64e11295c046 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.530835 Aug 21 01:31:52.867843 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 402/815] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:52 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:53.822305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4f0d4afc-4183-43cc-a631-bc727cf6f254 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:53.822305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4f0d4afc-4183-43cc-a631-bc727cf6f254 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:53.828061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f0d4afc-4183-43cc-a631-bc727cf6f254 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:53.828581 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f0d4afc-4183-43cc-a631-bc727cf6f254 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:53.829022 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f0d4afc-4183-43cc-a631-bc727cf6f254 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:53.829752 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f0d4afc-4183-43cc-a631-bc727cf6f254 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:53.830206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f0d4afc-4183-43cc-a631-bc727cf6f254 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:53.830626 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f0d4afc-4183-43cc-a631-bc727cf6f254 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:53.899058 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2475567e-542a-43ac-a13c-14798bbe1337 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:53.900374 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2475567e-542a-43ac-a13c-14798bbe1337 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:53.910333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2475567e-542a-43ac-a13c-14798bbe1337 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:53.911642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2475567e-542a-43ac-a13c-14798bbe1337 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:53.912497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2475567e-542a-43ac-a13c-14798bbe1337 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:53.913770 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2475567e-542a-43ac-a13c-14798bbe1337 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:53.914731 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2475567e-542a-43ac-a13c-14798bbe1337 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:53.915568 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2475567e-542a-43ac-a13c-14798bbe1337 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:53.982567 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4f0d4afc-4183-43cc-a631-bc727cf6f254 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:53.992498 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4f0d4afc-4183-43cc-a631-bc727cf6f254 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:53.994302 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4f0d4afc-4183-43cc-a631-bc727cf6f254 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:53.995472 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4f0d4afc-4183-43cc-a631-bc727cf6f254 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.179630 Aug 21 01:31:53.996015 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 414/816] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:53 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:54.176903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2475567e-542a-43ac-a13c-14798bbe1337 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:54.184817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2475567e-542a-43ac-a13c-14798bbe1337 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.292046 Aug 21 01:31:54.185396 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 403/817] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:53 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:55.022513 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-becd82c5-1c73-423e-9d8c-69008d76277b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:55.023089 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-becd82c5-1c73-423e-9d8c-69008d76277b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:55.029573 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-becd82c5-1c73-423e-9d8c-69008d76277b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:55.030065 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-becd82c5-1c73-423e-9d8c-69008d76277b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:55.030514 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-becd82c5-1c73-423e-9d8c-69008d76277b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:55.031208 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-becd82c5-1c73-423e-9d8c-69008d76277b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:55.031703 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-becd82c5-1c73-423e-9d8c-69008d76277b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:55.032156 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-becd82c5-1c73-423e-9d8c-69008d76277b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:55.163969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-becd82c5-1c73-423e-9d8c-69008d76277b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:55.172423 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-becd82c5-1c73-423e-9d8c-69008d76277b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:55.174410 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-becd82c5-1c73-423e-9d8c-69008d76277b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:55.175551 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-becd82c5-1c73-423e-9d8c-69008d76277b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.157671 Aug 21 01:31:55.176119 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 415/818] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:55 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:55.206263 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-040dad43-4a01-4148-8cb3-6f81f26fb1a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:55.206898 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-040dad43-4a01-4148-8cb3-6f81f26fb1a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:55.213494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-040dad43-4a01-4148-8cb3-6f81f26fb1a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:55.213974 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-040dad43-4a01-4148-8cb3-6f81f26fb1a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:55.214406 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-040dad43-4a01-4148-8cb3-6f81f26fb1a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:55.215094 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-040dad43-4a01-4148-8cb3-6f81f26fb1a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:55.215539 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-040dad43-4a01-4148-8cb3-6f81f26fb1a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:55.215966 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-040dad43-4a01-4148-8cb3-6f81f26fb1a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:55.352184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-040dad43-4a01-4148-8cb3-6f81f26fb1a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:55.359968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-040dad43-4a01-4148-8cb3-6f81f26fb1a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.157897 Aug 21 01:31:55.360570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 404/819] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:55 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:56.201334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-899fd598-63bf-4c8c-8973-34247b3d75a8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:56.202087 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-899fd598-63bf-4c8c-8973-34247b3d75a8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:56.210943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-899fd598-63bf-4c8c-8973-34247b3d75a8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:56.211647 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-899fd598-63bf-4c8c-8973-34247b3d75a8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:56.212280 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-899fd598-63bf-4c8c-8973-34247b3d75a8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:56.213298 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-899fd598-63bf-4c8c-8973-34247b3d75a8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:56.214301 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-899fd598-63bf-4c8c-8973-34247b3d75a8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:56.214517 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-899fd598-63bf-4c8c-8973-34247b3d75a8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:56.381999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e3eb3c40-5ef4-490b-821f-87e0836421f7 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:56.382882 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e3eb3c40-5ef4-490b-821f-87e0836421f7 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:56.389585 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e3eb3c40-5ef4-490b-821f-87e0836421f7 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:56.390074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e3eb3c40-5ef4-490b-821f-87e0836421f7 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:56.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e3eb3c40-5ef4-490b-821f-87e0836421f7 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:56.391241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e3eb3c40-5ef4-490b-821f-87e0836421f7 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:56.391846 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e3eb3c40-5ef4-490b-821f-87e0836421f7 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:56.392144 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e3eb3c40-5ef4-490b-821f-87e0836421f7 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:56.414601 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-899fd598-63bf-4c8c-8973-34247b3d75a8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:56.427379 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-899fd598-63bf-4c8c-8973-34247b3d75a8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:56.428964 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-899fd598-63bf-4c8c-8973-34247b3d75a8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:56.430140 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-899fd598-63bf-4c8c-8973-34247b3d75a8 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.234789 Aug 21 01:31:56.430700 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 416/820] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:56 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:56.545096 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e3eb3c40-5ef4-490b-821f-87e0836421f7 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:56.553696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e3eb3c40-5ef4-490b-821f-87e0836421f7 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.175477 Aug 21 01:31:56.554287 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 405/821] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:56 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:57.456893 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-46c5e4fc-f477-442d-9f85-b778bc061582 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:57.458033 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-46c5e4fc-f477-442d-9f85-b778bc061582 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:57.468295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-46c5e4fc-f477-442d-9f85-b778bc061582 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:57.469115 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-46c5e4fc-f477-442d-9f85-b778bc061582 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:57.469837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-46c5e4fc-f477-442d-9f85-b778bc061582 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:57.471007 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-46c5e4fc-f477-442d-9f85-b778bc061582 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:57.471774 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-46c5e4fc-f477-442d-9f85-b778bc061582 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:57.472554 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-46c5e4fc-f477-442d-9f85-b778bc061582 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:57.578165 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8030c706-4fb5-4372-ab07-813a9c365c2d tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:57.582639 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8030c706-4fb5-4372-ab07-813a9c365c2d tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:57.588594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8030c706-4fb5-4372-ab07-813a9c365c2d tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:57.589151 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8030c706-4fb5-4372-ab07-813a9c365c2d tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:57.589594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8030c706-4fb5-4372-ab07-813a9c365c2d tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:57.590377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8030c706-4fb5-4372-ab07-813a9c365c2d tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:57.590834 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8030c706-4fb5-4372-ab07-813a9c365c2d tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:57.591260 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8030c706-4fb5-4372-ab07-813a9c365c2d tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:57.727959 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-46c5e4fc-f477-442d-9f85-b778bc061582 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:57.734698 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-46c5e4fc-f477-442d-9f85-b778bc061582 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:57.736380 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-46c5e4fc-f477-442d-9f85-b778bc061582 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:57.738337 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-46c5e4fc-f477-442d-9f85-b778bc061582 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.288470 Aug 21 01:31:57.738901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 417/822] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:57 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:57.859600 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8030c706-4fb5-4372-ab07-813a9c365c2d tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:57.870317 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8030c706-4fb5-4372-ab07-813a9c365c2d tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.299035 Aug 21 01:31:57.886000 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 406/823] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:57 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:31:58.169275 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-fd5e7d38-9b4b-440d-a6f1-5ccf04dcd456 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:58.171956 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-fd5e7d38-9b4b-440d-a6f1-5ccf04dcd456 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:58.183800 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fd5e7d38-9b4b-440d-a6f1-5ccf04dcd456 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:58.184318 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fd5e7d38-9b4b-440d-a6f1-5ccf04dcd456 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:58.186880 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fd5e7d38-9b4b-440d-a6f1-5ccf04dcd456 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:58.186880 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fd5e7d38-9b4b-440d-a6f1-5ccf04dcd456 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:58.186880 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fd5e7d38-9b4b-440d-a6f1-5ccf04dcd456 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:58.186880 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fd5e7d38-9b4b-440d-a6f1-5ccf04dcd456 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:58.232948 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-fd5e7d38-9b4b-440d-a6f1-5ccf04dcd456 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.067896 Aug 21 01:31:58.233966 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 418/824] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:31:58 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments => generated 371 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:58.390619 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-cd1389c4-5571-4ae9-92d2-82b44df60638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:58.391505 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-cd1389c4-5571-4ae9-92d2-82b44df60638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:58.398873 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cd1389c4-5571-4ae9-92d2-82b44df60638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:58.399814 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cd1389c4-5571-4ae9-92d2-82b44df60638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:58.400487 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cd1389c4-5571-4ae9-92d2-82b44df60638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:58.401529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cd1389c4-5571-4ae9-92d2-82b44df60638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:58.402158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cd1389c4-5571-4ae9-92d2-82b44df60638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:58.402774 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cd1389c4-5571-4ae9-92d2-82b44df60638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:58.458735 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-cd1389c4-5571-4ae9-92d2-82b44df60638 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments/30c9c194-809c-4a6e-b408-a7b8d79a3d5c" status: 200 len: 194 microversion: 2.1 time: 0.072073 Aug 21 01:31:58.460034 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 407/825] 10.222.0.47 () {58 vars in 1413 bytes} [Sun Aug 21 01:31:58 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments/30c9c194-809c-4a6e-b408-a7b8d79a3d5c => generated 194 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:31:58.471234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-27d60f8f-54ed-4f7e-be52-282ca6251e20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:58.472065 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-27d60f8f-54ed-4f7e-be52-282ca6251e20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:58.478651 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-27d60f8f-54ed-4f7e-be52-282ca6251e20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:58.479145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-27d60f8f-54ed-4f7e-be52-282ca6251e20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:58.479584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-27d60f8f-54ed-4f7e-be52-282ca6251e20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:58.480347 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-27d60f8f-54ed-4f7e-be52-282ca6251e20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:58.480801 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-27d60f8f-54ed-4f7e-be52-282ca6251e20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:58.481237 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-27d60f8f-54ed-4f7e-be52-282ca6251e20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:58.607942 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-27d60f8f-54ed-4f7e-be52-282ca6251e20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lazy-loading 'services' on Instance uuid 9a075e00-151c-4ac4-b454-deea3de38ef7 {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:31:58.769579 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fd42321d-035d-4b6b-b57e-9de141834a13 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:58.771212 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fd42321d-035d-4b6b-b57e-9de141834a13 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:58.780220 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd42321d-035d-4b6b-b57e-9de141834a13 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:58.780969 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd42321d-035d-4b6b-b57e-9de141834a13 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:58.781562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd42321d-035d-4b6b-b57e-9de141834a13 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:58.782462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd42321d-035d-4b6b-b57e-9de141834a13 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:58.783999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd42321d-035d-4b6b-b57e-9de141834a13 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:58.784655 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fd42321d-035d-4b6b-b57e-9de141834a13 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:58.872230 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-27d60f8f-54ed-4f7e-be52-282ca6251e20 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "DELETE /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments/30c9c194-809c-4a6e-b408-a7b8d79a3d5c" status: 202 len: 0 microversion: 2.1 time: 0.404997 Aug 21 01:31:58.879612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 419/826] 10.222.0.47 () {58 vars in 1416 bytes} [Sun Aug 21 01:31:58 2022] DELETE /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments/30c9c194-809c-4a6e-b408-a7b8d79a3d5c => generated 0 bytes in 414 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 21 01:31:58.895759 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-fe645679-aabe-46d5-b6a5-0104f6b67fe4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:58.915982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-fe645679-aabe-46d5-b6a5-0104f6b67fe4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:58.935882 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fe645679-aabe-46d5-b6a5-0104f6b67fe4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:58.939911 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fe645679-aabe-46d5-b6a5-0104f6b67fe4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:58.940746 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fe645679-aabe-46d5-b6a5-0104f6b67fe4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:58.942657 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fe645679-aabe-46d5-b6a5-0104f6b67fe4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:58.943457 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fe645679-aabe-46d5-b6a5-0104f6b67fe4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:58.945158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-fe645679-aabe-46d5-b6a5-0104f6b67fe4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:59.020891 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fd42321d-035d-4b6b-b57e-9de141834a13 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:59.032440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fd42321d-035d-4b6b-b57e-9de141834a13 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:59.035009 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fd42321d-035d-4b6b-b57e-9de141834a13 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:59.036576 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fd42321d-035d-4b6b-b57e-9de141834a13 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.278116 Aug 21 01:31:59.037693 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 408/827] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:31:58 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:31:59.195642 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-fe645679-aabe-46d5-b6a5-0104f6b67fe4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:59.205855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-fe645679-aabe-46d5-b6a5-0104f6b67fe4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1569 microversion: 2.1 time: 0.314718 Aug 21 01:31:59.207085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 420/828] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:58 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1569 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:31:59.229471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a48f8fb6-6378-4466-83f5-7432e05a0324 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:59.230162 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a48f8fb6-6378-4466-83f5-7432e05a0324 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:59.241094 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a48f8fb6-6378-4466-83f5-7432e05a0324 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:59.241855 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a48f8fb6-6378-4466-83f5-7432e05a0324 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:59.253665 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a48f8fb6-6378-4466-83f5-7432e05a0324 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:59.254070 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a48f8fb6-6378-4466-83f5-7432e05a0324 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:59.254831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a48f8fb6-6378-4466-83f5-7432e05a0324 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:59.255031 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a48f8fb6-6378-4466-83f5-7432e05a0324 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:59.297936 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a48f8fb6-6378-4466-83f5-7432e05a0324 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=121276) unshelve /opt/stack/nova/nova/compute/api.py:4578}} Aug 21 01:31:59.395213 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-fe72e3ec-430b-4a16-96c4-689a6e385454 req-5bde6582-7467-496a-967a-f954a33c0a74 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "119204ec-e0f8-439e-b985-b934d251684f", "name": "network-vif-unplugged", "status": "completed", "tag": "bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:31:59.405915 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-fe72e3ec-430b-4a16-96c4-689a6e385454 req-5bde6582-7467-496a-967a-f954a33c0a74 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:59.406393 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-fe72e3ec-430b-4a16-96c4-689a6e385454 req-5bde6582-7467-496a-967a-f954a33c0a74 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:59.406820 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-fe72e3ec-430b-4a16-96c4-689a6e385454 req-5bde6582-7467-496a-967a-f954a33c0a74 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:59.436607 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-fe72e3ec-430b-4a16-96c4-689a6e385454 req-5bde6582-7467-496a-967a-f954a33c0a74 service nova] Creating event network-vif-unplugged:bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f for instance 119204ec-e0f8-439e-b985-b934d251684f on hv-cinder-maste Aug 21 01:31:59.442325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-fe72e3ec-430b-4a16-96c4-689a6e385454 req-5bde6582-7467-496a-967a-f954a33c0a74 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.054528 Aug 21 01:31:59.442985 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 421/829] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:31:59 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:31:59.622658 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a48f8fb6-6378-4466-83f5-7432e05a0324 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "POST /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f/action" status: 202 len: 0 microversion: 2.1 time: 0.397158 Aug 21 01:31:59.626756 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 409/830] 10.222.0.47 () {60 vars in 1278 bytes} [Sun Aug 21 01:31:59 2022] POST /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f/action => generated 0 bytes in 400 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:31:59.634595 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2d2afc3c-ced9-4a87-ab7a-3b21f4ca9315 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:31:59.635931 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-2d2afc3c-ced9-4a87-ab7a-3b21f4ca9315 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:31:59.643506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2d2afc3c-ced9-4a87-ab7a-3b21f4ca9315 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:59.644006 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2d2afc3c-ced9-4a87-ab7a-3b21f4ca9315 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:59.644469 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2d2afc3c-ced9-4a87-ab7a-3b21f4ca9315 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:59.645141 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2d2afc3c-ced9-4a87-ab7a-3b21f4ca9315 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:31:59.645653 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2d2afc3c-ced9-4a87-ab7a-3b21f4ca9315 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:31:59.646042 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2d2afc3c-ced9-4a87-ab7a-3b21f4ca9315 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:31:59.812774 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2d2afc3c-ced9-4a87-ab7a-3b21f4ca9315 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:31:59.828060 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2d2afc3c-ced9-4a87-ab7a-3b21f4ca9315 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1577 microversion: 2.1 time: 0.198276 Aug 21 01:31:59.830590 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 422/831] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:31:59 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1577 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:00.062684 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-aa0bc7ad-1265-4193-96db-10abbd24fd03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:00.063830 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-aa0bc7ad-1265-4193-96db-10abbd24fd03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:00.085186 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aa0bc7ad-1265-4193-96db-10abbd24fd03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:00.087324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aa0bc7ad-1265-4193-96db-10abbd24fd03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:00.089423 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aa0bc7ad-1265-4193-96db-10abbd24fd03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:00.092369 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aa0bc7ad-1265-4193-96db-10abbd24fd03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:00.093141 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aa0bc7ad-1265-4193-96db-10abbd24fd03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:00.093886 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aa0bc7ad-1265-4193-96db-10abbd24fd03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:00.326103 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-aa0bc7ad-1265-4193-96db-10abbd24fd03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:00.344326 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-aa0bc7ad-1265-4193-96db-10abbd24fd03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:00.353129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-aa0bc7ad-1265-4193-96db-10abbd24fd03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:00.355297 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-aa0bc7ad-1265-4193-96db-10abbd24fd03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.299190 Aug 21 01:32:00.356883 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 410/832] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:00 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:00.857535 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c2831e39-5982-4d37-b254-905a05c71efd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:00.858810 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c2831e39-5982-4d37-b254-905a05c71efd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:00.866023 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c2831e39-5982-4d37-b254-905a05c71efd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:00.866519 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c2831e39-5982-4d37-b254-905a05c71efd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:00.866978 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c2831e39-5982-4d37-b254-905a05c71efd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:00.867870 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c2831e39-5982-4d37-b254-905a05c71efd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:00.868380 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c2831e39-5982-4d37-b254-905a05c71efd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:00.868809 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c2831e39-5982-4d37-b254-905a05c71efd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:01.027381 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c2831e39-5982-4d37-b254-905a05c71efd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:01.036088 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c2831e39-5982-4d37-b254-905a05c71efd tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1577 microversion: 2.1 time: 0.183101 Aug 21 01:32:01.039529 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 423/833] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:00 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1577 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:01.381499 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-09d82647-661d-4431-8402-34ee05901d72 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:01.383299 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-09d82647-661d-4431-8402-34ee05901d72 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:01.395632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-09d82647-661d-4431-8402-34ee05901d72 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:01.396164 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-09d82647-661d-4431-8402-34ee05901d72 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:01.398848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-09d82647-661d-4431-8402-34ee05901d72 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:01.398848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-09d82647-661d-4431-8402-34ee05901d72 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:01.398848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-09d82647-661d-4431-8402-34ee05901d72 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:01.398848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-09d82647-661d-4431-8402-34ee05901d72 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:01.563957 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-09d82647-661d-4431-8402-34ee05901d72 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:01.572429 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-09d82647-661d-4431-8402-34ee05901d72 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:01.573955 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-09d82647-661d-4431-8402-34ee05901d72 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:01.575128 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-09d82647-661d-4431-8402-34ee05901d72 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.199394 Aug 21 01:32:01.575870 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 411/834] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:01 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:02.063890 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d9347ca1-ea7c-4094-bcbf-b019f0c2a312 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:02.064999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d9347ca1-ea7c-4094-bcbf-b019f0c2a312 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:02.076020 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d9347ca1-ea7c-4094-bcbf-b019f0c2a312 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:02.076949 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d9347ca1-ea7c-4094-bcbf-b019f0c2a312 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:02.077756 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d9347ca1-ea7c-4094-bcbf-b019f0c2a312 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:02.078614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d9347ca1-ea7c-4094-bcbf-b019f0c2a312 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:02.079279 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d9347ca1-ea7c-4094-bcbf-b019f0c2a312 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:02.079949 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d9347ca1-ea7c-4094-bcbf-b019f0c2a312 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:02.257001 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d9347ca1-ea7c-4094-bcbf-b019f0c2a312 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:02.298020 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d9347ca1-ea7c-4094-bcbf-b019f0c2a312 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1577 microversion: 2.1 time: 0.241660 Aug 21 01:32:02.304977 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 424/835] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:02 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1577 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:32:02.599717 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ae68b07a-063a-4269-a668-31f75fee242a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:02.600697 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ae68b07a-063a-4269-a668-31f75fee242a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:02.607484 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae68b07a-063a-4269-a668-31f75fee242a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:02.608038 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae68b07a-063a-4269-a668-31f75fee242a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:02.608494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae68b07a-063a-4269-a668-31f75fee242a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:02.609198 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae68b07a-063a-4269-a668-31f75fee242a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:02.609655 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae68b07a-063a-4269-a668-31f75fee242a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:02.610074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ae68b07a-063a-4269-a668-31f75fee242a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:02.754549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ae68b07a-063a-4269-a668-31f75fee242a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:02.763623 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ae68b07a-063a-4269-a668-31f75fee242a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:02.765167 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ae68b07a-063a-4269-a668-31f75fee242a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:02.766604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ae68b07a-063a-4269-a668-31f75fee242a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.171489 Aug 21 01:32:02.767163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 412/836] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:02 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:03.325884 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-747ae776-38a1-4e24-b6da-8f22f63dd0d2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:03.326512 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-747ae776-38a1-4e24-b6da-8f22f63dd0d2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:03.342732 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-747ae776-38a1-4e24-b6da-8f22f63dd0d2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:03.343377 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-747ae776-38a1-4e24-b6da-8f22f63dd0d2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:03.343837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-747ae776-38a1-4e24-b6da-8f22f63dd0d2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:03.356201 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-747ae776-38a1-4e24-b6da-8f22f63dd0d2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:03.357117 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-747ae776-38a1-4e24-b6da-8f22f63dd0d2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:03.357752 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-747ae776-38a1-4e24-b6da-8f22f63dd0d2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:03.563653 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-747ae776-38a1-4e24-b6da-8f22f63dd0d2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:03.572846 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-747ae776-38a1-4e24-b6da-8f22f63dd0d2 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1577 microversion: 2.1 time: 0.250836 Aug 21 01:32:03.573769 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 425/837] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:03 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1577 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:03.790958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6c585399-359c-47d8-9db1-2557b819fc3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:03.792410 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6c585399-359c-47d8-9db1-2557b819fc3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:03.796307 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=68,nova_api:UPDATE=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:03.801007 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c585399-359c-47d8-9db1-2557b819fc3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:03.804736 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c585399-359c-47d8-9db1-2557b819fc3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:03.805306 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c585399-359c-47d8-9db1-2557b819fc3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:03.806359 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c585399-359c-47d8-9db1-2557b819fc3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:03.806925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c585399-359c-47d8-9db1-2557b819fc3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:03.807890 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6c585399-359c-47d8-9db1-2557b819fc3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:03.975864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6c585399-359c-47d8-9db1-2557b819fc3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:03.985419 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6c585399-359c-47d8-9db1-2557b819fc3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:03.987719 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6c585399-359c-47d8-9db1-2557b819fc3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:03.989098 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6c585399-359c-47d8-9db1-2557b819fc3e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.202250 Aug 21 01:32:03.989790 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 413/838] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:03 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:32:04.594880 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-14465560-3f8d-4352-9384-106a055a8699 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:04.596272 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-14465560-3f8d-4352-9384-106a055a8699 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:04.603682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-14465560-3f8d-4352-9384-106a055a8699 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:04.604586 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-14465560-3f8d-4352-9384-106a055a8699 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:04.605173 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-14465560-3f8d-4352-9384-106a055a8699 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:04.606045 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-14465560-3f8d-4352-9384-106a055a8699 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:04.606840 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-14465560-3f8d-4352-9384-106a055a8699 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:04.607590 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-14465560-3f8d-4352-9384-106a055a8699 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:04.777264 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-14465560-3f8d-4352-9384-106a055a8699 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:04.785670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-14465560-3f8d-4352-9384-106a055a8699 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1577 microversion: 2.1 time: 0.194806 Aug 21 01:32:04.786275 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 426/839] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:04 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1577 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:05.011897 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9f29d8ae-e42c-44c4-bc61-2299bf71d18b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:05.012541 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9f29d8ae-e42c-44c4-bc61-2299bf71d18b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:05.027610 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9f29d8ae-e42c-44c4-bc61-2299bf71d18b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:05.029360 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9f29d8ae-e42c-44c4-bc61-2299bf71d18b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:05.029360 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9f29d8ae-e42c-44c4-bc61-2299bf71d18b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:05.029585 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9f29d8ae-e42c-44c4-bc61-2299bf71d18b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:05.029984 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9f29d8ae-e42c-44c4-bc61-2299bf71d18b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:05.030599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9f29d8ae-e42c-44c4-bc61-2299bf71d18b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:05.181513 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9f29d8ae-e42c-44c4-bc61-2299bf71d18b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:05.192005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9f29d8ae-e42c-44c4-bc61-2299bf71d18b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:05.193922 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9f29d8ae-e42c-44c4-bc61-2299bf71d18b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:05.195090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9f29d8ae-e42c-44c4-bc61-2299bf71d18b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.187283 Aug 21 01:32:05.196049 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 414/840] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:05 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:05.836425 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c2281b64-fab9-4269-8ded-1a43cc81acf9 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:05.836425 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c2281b64-fab9-4269-8ded-1a43cc81acf9 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:05.836425 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c2281b64-fab9-4269-8ded-1a43cc81acf9 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:05.836425 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c2281b64-fab9-4269-8ded-1a43cc81acf9 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:05.837454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c2281b64-fab9-4269-8ded-1a43cc81acf9 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:05.837454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c2281b64-fab9-4269-8ded-1a43cc81acf9 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:05.837454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c2281b64-fab9-4269-8ded-1a43cc81acf9 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:05.837454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c2281b64-fab9-4269-8ded-1a43cc81acf9 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:06.006661 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c2281b64-fab9-4269-8ded-1a43cc81acf9 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:06.015475 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c2281b64-fab9-4269-8ded-1a43cc81acf9 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1577 microversion: 2.1 time: 0.208145 Aug 21 01:32:06.016157 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 427/841] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:05 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1577 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:06.219608 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1f3dfe75-0bab-4fce-be5e-6c03b74332bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:06.220223 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1f3dfe75-0bab-4fce-be5e-6c03b74332bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:06.226696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1f3dfe75-0bab-4fce-be5e-6c03b74332bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:06.227170 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1f3dfe75-0bab-4fce-be5e-6c03b74332bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:06.227631 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1f3dfe75-0bab-4fce-be5e-6c03b74332bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:06.228453 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1f3dfe75-0bab-4fce-be5e-6c03b74332bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:06.228909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1f3dfe75-0bab-4fce-be5e-6c03b74332bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:06.229328 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1f3dfe75-0bab-4fce-be5e-6c03b74332bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:06.366150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1f3dfe75-0bab-4fce-be5e-6c03b74332bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:06.369619 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:UPDATE=2,nova_cell1:SELECT=96,nova_cell1:INSERT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:06.374714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1f3dfe75-0bab-4fce-be5e-6c03b74332bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:06.376173 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1f3dfe75-0bab-4fce-be5e-6c03b74332bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:06.378020 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1f3dfe75-0bab-4fce-be5e-6c03b74332bd tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.162212 Aug 21 01:32:06.378577 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 415/842] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:06 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:07.038465 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7be336ba-599f-43ad-9e53-484e9b12cae5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:07.039108 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7be336ba-599f-43ad-9e53-484e9b12cae5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:07.045603 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7be336ba-599f-43ad-9e53-484e9b12cae5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:07.046145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7be336ba-599f-43ad-9e53-484e9b12cae5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:07.046580 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7be336ba-599f-43ad-9e53-484e9b12cae5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:07.047276 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7be336ba-599f-43ad-9e53-484e9b12cae5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:07.047724 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7be336ba-599f-43ad-9e53-484e9b12cae5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:07.048173 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7be336ba-599f-43ad-9e53-484e9b12cae5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:07.064118 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:UPDATE=1,nova_cell1:SELECT=98,nova_cell1:INSERT=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:07.183051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7be336ba-599f-43ad-9e53-484e9b12cae5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:07.190795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7be336ba-599f-43ad-9e53-484e9b12cae5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1577 microversion: 2.1 time: 0.156448 Aug 21 01:32:07.197478 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 428/843] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:07 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1577 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:07.403016 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-27cad9e2-6022-494d-aa55-f0f7fc617875 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:07.403814 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-27cad9e2-6022-494d-aa55-f0f7fc617875 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:07.411724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-27cad9e2-6022-494d-aa55-f0f7fc617875 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:07.412237 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-27cad9e2-6022-494d-aa55-f0f7fc617875 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:07.412695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-27cad9e2-6022-494d-aa55-f0f7fc617875 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:07.413403 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-27cad9e2-6022-494d-aa55-f0f7fc617875 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:07.413848 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-27cad9e2-6022-494d-aa55-f0f7fc617875 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:07.414267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-27cad9e2-6022-494d-aa55-f0f7fc617875 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:07.549946 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-27cad9e2-6022-494d-aa55-f0f7fc617875 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:07.558355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-27cad9e2-6022-494d-aa55-f0f7fc617875 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:07.559919 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-27cad9e2-6022-494d-aa55-f0f7fc617875 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:07.561138 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-27cad9e2-6022-494d-aa55-f0f7fc617875 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.162864 Aug 21 01:32:07.561707 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 416/844] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:07 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:32:08.212803 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7250a626-beca-474e-b9df-f1932b9b7853 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:08.213560 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7250a626-beca-474e-b9df-f1932b9b7853 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:08.220534 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7250a626-beca-474e-b9df-f1932b9b7853 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:08.221020 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7250a626-beca-474e-b9df-f1932b9b7853 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:08.221457 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7250a626-beca-474e-b9df-f1932b9b7853 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:08.222194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7250a626-beca-474e-b9df-f1932b9b7853 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:08.223720 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7250a626-beca-474e-b9df-f1932b9b7853 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:08.223720 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7250a626-beca-474e-b9df-f1932b9b7853 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:08.387661 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7250a626-beca-474e-b9df-f1932b9b7853 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:08.396555 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7250a626-beca-474e-b9df-f1932b9b7853 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1517 microversion: 2.1 time: 0.187376 Aug 21 01:32:08.396555 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 429/845] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:08 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1517 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:08.585698 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-35d3ed9e-510b-4e11-9ced-555b320c0df6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:08.586221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-35d3ed9e-510b-4e11-9ced-555b320c0df6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:08.592745 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-35d3ed9e-510b-4e11-9ced-555b320c0df6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:08.593221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-35d3ed9e-510b-4e11-9ced-555b320c0df6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:08.593653 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-35d3ed9e-510b-4e11-9ced-555b320c0df6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:08.594345 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-35d3ed9e-510b-4e11-9ced-555b320c0df6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:08.594931 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-35d3ed9e-510b-4e11-9ced-555b320c0df6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:08.595439 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-35d3ed9e-510b-4e11-9ced-555b320c0df6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:08.740545 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-35d3ed9e-510b-4e11-9ced-555b320c0df6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:08.740545 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-35d3ed9e-510b-4e11-9ced-555b320c0df6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:08.740852 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-35d3ed9e-510b-4e11-9ced-555b320c0df6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:08.741608 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-35d3ed9e-510b-4e11-9ced-555b320c0df6 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.160661 Aug 21 01:32:08.742167 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 417/846] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:08 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:32:09.417403 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5815a468-d850-4bf9-9163-8ccd62c15827 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:09.417953 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5815a468-d850-4bf9-9163-8ccd62c15827 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:09.424807 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5815a468-d850-4bf9-9163-8ccd62c15827 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:09.425305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5815a468-d850-4bf9-9163-8ccd62c15827 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:09.425744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5815a468-d850-4bf9-9163-8ccd62c15827 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:09.426978 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5815a468-d850-4bf9-9163-8ccd62c15827 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:09.427455 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5815a468-d850-4bf9-9163-8ccd62c15827 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:09.427866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5815a468-d850-4bf9-9163-8ccd62c15827 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:09.572781 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5815a468-d850-4bf9-9163-8ccd62c15827 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:09.580982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5815a468-d850-4bf9-9163-8ccd62c15827 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1519 microversion: 2.1 time: 0.167756 Aug 21 01:32:09.581553 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 430/847] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:09 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1519 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:09.764917 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-aaf3db79-891e-4ece-bc2b-511f430f1dec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:09.772085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-aaf3db79-891e-4ece-bc2b-511f430f1dec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:09.772471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aaf3db79-891e-4ece-bc2b-511f430f1dec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:09.772897 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aaf3db79-891e-4ece-bc2b-511f430f1dec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:09.773334 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aaf3db79-891e-4ece-bc2b-511f430f1dec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:09.774214 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aaf3db79-891e-4ece-bc2b-511f430f1dec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:09.774479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aaf3db79-891e-4ece-bc2b-511f430f1dec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:09.774900 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-aaf3db79-891e-4ece-bc2b-511f430f1dec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:09.808931 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-0bca8ecc-3a3a-47e7-a95f-6dd9a5a8a16a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:09.809478 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-0bca8ecc-3a3a-47e7-a95f-6dd9a5a8a16a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:09.815605 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0bca8ecc-3a3a-47e7-a95f-6dd9a5a8a16a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:09.816094 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0bca8ecc-3a3a-47e7-a95f-6dd9a5a8a16a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:09.816557 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0bca8ecc-3a3a-47e7-a95f-6dd9a5a8a16a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:09.817363 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0bca8ecc-3a3a-47e7-a95f-6dd9a5a8a16a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:09.817732 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0bca8ecc-3a3a-47e7-a95f-6dd9a5a8a16a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:09.818333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0bca8ecc-3a3a-47e7-a95f-6dd9a5a8a16a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:09.859358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-0bca8ecc-3a3a-47e7-a95f-6dd9a5a8a16a tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments/ccf43391-fddc-43c3-b309-1561379e4aa1" status: 200 len: 194 microversion: 2.1 time: 0.054027 Aug 21 01:32:09.863567 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 431/848] 10.222.0.47 () {58 vars in 1413 bytes} [Sun Aug 21 01:32:09 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments/ccf43391-fddc-43c3-b309-1561379e4aa1 => generated 194 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:32:09.875999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4cabc250-0bf1-4fb4-b760-ea39a3716ab5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:09.876698 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4cabc250-0bf1-4fb4-b760-ea39a3716ab5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:09.886930 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4cabc250-0bf1-4fb4-b760-ea39a3716ab5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:09.886930 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4cabc250-0bf1-4fb4-b760-ea39a3716ab5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:09.886930 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4cabc250-0bf1-4fb4-b760-ea39a3716ab5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:09.886930 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4cabc250-0bf1-4fb4-b760-ea39a3716ab5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:09.887446 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4cabc250-0bf1-4fb4-b760-ea39a3716ab5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:09.887446 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4cabc250-0bf1-4fb4-b760-ea39a3716ab5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:09.926034 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-aaf3db79-891e-4ece-bc2b-511f430f1dec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:09.935812 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-aaf3db79-891e-4ece-bc2b-511f430f1dec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:09.937739 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-aaf3db79-891e-4ece-bc2b-511f430f1dec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:09.938932 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-aaf3db79-891e-4ece-bc2b-511f430f1dec tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.177969 Aug 21 01:32:09.939493 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 418/849] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:09 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:09.985558 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-4cabc250-0bf1-4fb4-b760-ea39a3716ab5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lazy-loading 'services' on Instance uuid 9a075e00-151c-4ac4-b454-deea3de38ef7 {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:32:10.276080 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4cabc250-0bf1-4fb4-b760-ea39a3716ab5 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "DELETE /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments/ccf43391-fddc-43c3-b309-1561379e4aa1" status: 202 len: 0 microversion: 2.1 time: 0.403930 Aug 21 01:32:10.277295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 432/850] 10.222.0.47 () {58 vars in 1416 bytes} [Sun Aug 21 01:32:09 2022] DELETE /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments/ccf43391-fddc-43c3-b309-1561379e4aa1 => generated 0 bytes in 406 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 21 01:32:10.608664 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-825d0d24-f251-4928-bd8c-f4729b32297c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:10.609312 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-825d0d24-f251-4928-bd8c-f4729b32297c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:10.619226 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-825d0d24-f251-4928-bd8c-f4729b32297c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:10.619733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-825d0d24-f251-4928-bd8c-f4729b32297c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:10.620205 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-825d0d24-f251-4928-bd8c-f4729b32297c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:10.620927 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-825d0d24-f251-4928-bd8c-f4729b32297c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:10.621417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-825d0d24-f251-4928-bd8c-f4729b32297c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:10.621854 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-825d0d24-f251-4928-bd8c-f4729b32297c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:10.764080 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-825d0d24-f251-4928-bd8c-f4729b32297c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:10.773188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-825d0d24-f251-4928-bd8c-f4729b32297c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1519 microversion: 2.1 time: 0.169001 Aug 21 01:32:10.773960 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 419/851] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:10 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1519 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:32:10.963815 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a65a2f52-1646-48ea-b50c-b6874c49541f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:10.964753 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a65a2f52-1646-48ea-b50c-b6874c49541f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:10.971540 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a65a2f52-1646-48ea-b50c-b6874c49541f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:10.972273 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a65a2f52-1646-48ea-b50c-b6874c49541f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:10.972725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a65a2f52-1646-48ea-b50c-b6874c49541f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:10.973570 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a65a2f52-1646-48ea-b50c-b6874c49541f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:10.974084 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a65a2f52-1646-48ea-b50c-b6874c49541f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:10.974562 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a65a2f52-1646-48ea-b50c-b6874c49541f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:11.134251 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a65a2f52-1646-48ea-b50c-b6874c49541f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:11.142916 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a65a2f52-1646-48ea-b50c-b6874c49541f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:11.144475 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a65a2f52-1646-48ea-b50c-b6874c49541f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:11.145655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a65a2f52-1646-48ea-b50c-b6874c49541f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.187336 Aug 21 01:32:11.146345 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 433/852] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:10 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:11.529448 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-a4004bcb-ce5b-4855-a192-a44b6a6ec1dd req-fb94a238-b702-44b5-be54-5546e7f1bce8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a5b475aa-486a-4957-8b54-aa1ee7438a8a", "name": "network-vif-plugged", "status": "completed", "tag": "d3beeb80-5980-4de1-b832-2d4a1e1dae1c"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:32:11.540025 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-a4004bcb-ce5b-4855-a192-a44b6a6ec1dd req-fb94a238-b702-44b5-be54-5546e7f1bce8 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:11.540536 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-a4004bcb-ce5b-4855-a192-a44b6a6ec1dd req-fb94a238-b702-44b5-be54-5546e7f1bce8 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:11.540965 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-a4004bcb-ce5b-4855-a192-a44b6a6ec1dd req-fb94a238-b702-44b5-be54-5546e7f1bce8 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:11.552558 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-a4004bcb-ce5b-4855-a192-a44b6a6ec1dd req-fb94a238-b702-44b5-be54-5546e7f1bce8 service nova] Creating event network-vif-plugged:d3beeb80-5980-4de1-b832-2d4a1e1dae1c for instance a5b475aa-486a-4957-8b54-aa1ee7438a8a on hv-cinder-maste Aug 21 01:32:11.557664 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-a4004bcb-ce5b-4855-a192-a44b6a6ec1dd req-fb94a238-b702-44b5-be54-5546e7f1bce8 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036097 Aug 21 01:32:11.558257 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 420/853] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:32:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:32:11.794216 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6d0c7f78-2f3b-443c-a726-7d6b9a7e14b6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:11.794828 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6d0c7f78-2f3b-443c-a726-7d6b9a7e14b6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:11.800855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6d0c7f78-2f3b-443c-a726-7d6b9a7e14b6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:11.801427 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6d0c7f78-2f3b-443c-a726-7d6b9a7e14b6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:11.801867 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6d0c7f78-2f3b-443c-a726-7d6b9a7e14b6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:11.802553 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6d0c7f78-2f3b-443c-a726-7d6b9a7e14b6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:11.803010 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6d0c7f78-2f3b-443c-a726-7d6b9a7e14b6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:11.803431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6d0c7f78-2f3b-443c-a726-7d6b9a7e14b6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:11.950145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6d0c7f78-2f3b-443c-a726-7d6b9a7e14b6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:11.959460 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6d0c7f78-2f3b-443c-a726-7d6b9a7e14b6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1519 microversion: 2.1 time: 0.168933 Aug 21 01:32:11.960153 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 434/854] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:11 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1519 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:12.173549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a711c21b-decd-438b-b77a-6fde7c0ed1bf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:12.174312 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a711c21b-decd-438b-b77a-6fde7c0ed1bf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:12.180940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a711c21b-decd-438b-b77a-6fde7c0ed1bf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:12.181432 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a711c21b-decd-438b-b77a-6fde7c0ed1bf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:12.181875 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a711c21b-decd-438b-b77a-6fde7c0ed1bf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:12.182581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a711c21b-decd-438b-b77a-6fde7c0ed1bf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:12.183174 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a711c21b-decd-438b-b77a-6fde7c0ed1bf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:12.183586 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a711c21b-decd-438b-b77a-6fde7c0ed1bf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:12.338281 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a711c21b-decd-438b-b77a-6fde7c0ed1bf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:12.348894 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a711c21b-decd-438b-b77a-6fde7c0ed1bf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:12.351072 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a711c21b-decd-438b-b77a-6fde7c0ed1bf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:12.352133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a711c21b-decd-438b-b77a-6fde7c0ed1bf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.183976 Aug 21 01:32:12.352932 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 421/855] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:12 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:12.981227 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5ee3c941-ebd2-485f-a862-fd40eb3490ed tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:12.981844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5ee3c941-ebd2-485f-a862-fd40eb3490ed tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:12.988492 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5ee3c941-ebd2-485f-a862-fd40eb3490ed tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:12.989580 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5ee3c941-ebd2-485f-a862-fd40eb3490ed tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:12.990022 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5ee3c941-ebd2-485f-a862-fd40eb3490ed tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:12.990734 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5ee3c941-ebd2-485f-a862-fd40eb3490ed tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:12.991196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5ee3c941-ebd2-485f-a862-fd40eb3490ed tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:12.991624 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5ee3c941-ebd2-485f-a862-fd40eb3490ed tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:13.132408 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5ee3c941-ebd2-485f-a862-fd40eb3490ed tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:13.144162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5ee3c941-ebd2-485f-a862-fd40eb3490ed tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.166489 Aug 21 01:32:13.152869 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 435/856] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:12 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:13.377315 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-43e84f75-5cfd-4075-9444-41400883d431 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:13.377997 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-43e84f75-5cfd-4075-9444-41400883d431 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:13.385673 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-43e84f75-5cfd-4075-9444-41400883d431 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:13.386230 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-43e84f75-5cfd-4075-9444-41400883d431 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:13.386780 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-43e84f75-5cfd-4075-9444-41400883d431 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:13.387563 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-43e84f75-5cfd-4075-9444-41400883d431 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:13.388045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-43e84f75-5cfd-4075-9444-41400883d431 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:13.388502 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-43e84f75-5cfd-4075-9444-41400883d431 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:13.529921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-43e84f75-5cfd-4075-9444-41400883d431 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:13.538516 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-43e84f75-5cfd-4075-9444-41400883d431 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:13.540348 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-43e84f75-5cfd-4075-9444-41400883d431 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:13.541512 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-43e84f75-5cfd-4075-9444-41400883d431 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.168329 Aug 21 01:32:13.542068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 422/857] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:13 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:13.579271 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-71d41bf2-8c3e-49a9-abf1-cbdf613ea185 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:13.581583 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-71d41bf2-8c3e-49a9-abf1-cbdf613ea185 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:13.588343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-71d41bf2-8c3e-49a9-abf1-cbdf613ea185 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:13.588862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-71d41bf2-8c3e-49a9-abf1-cbdf613ea185 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:13.589279 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-71d41bf2-8c3e-49a9-abf1-cbdf613ea185 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:13.589985 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-71d41bf2-8c3e-49a9-abf1-cbdf613ea185 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:13.590443 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-71d41bf2-8c3e-49a9-abf1-cbdf613ea185 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:13.590865 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-71d41bf2-8c3e-49a9-abf1-cbdf613ea185 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:13.621553 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-71d41bf2-8c3e-49a9-abf1-cbdf613ea185 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046467 Aug 21 01:32:13.622321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 436/858] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:32:13 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 21 01:32:13.738053 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bfd9e120-590d-4fd6-8370-997e824a2757 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:13.740675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bfd9e120-590d-4fd6-8370-997e824a2757 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:13.746674 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bfd9e120-590d-4fd6-8370-997e824a2757 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:13.747156 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bfd9e120-590d-4fd6-8370-997e824a2757 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:13.747593 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bfd9e120-590d-4fd6-8370-997e824a2757 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:13.748311 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bfd9e120-590d-4fd6-8370-997e824a2757 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:13.748764 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bfd9e120-590d-4fd6-8370-997e824a2757 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:13.749185 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bfd9e120-590d-4fd6-8370-997e824a2757 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:13.778906 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bfd9e120-590d-4fd6-8370-997e824a2757 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044925 Aug 21 01:32:13.779487 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 423/859] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:32:13 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 21 01:32:13.842489 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6d4e66f7-c195-4cd7-b842-e12dc53712e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:13.843052 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6d4e66f7-c195-4cd7-b842-e12dc53712e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:13.849152 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6d4e66f7-c195-4cd7-b842-e12dc53712e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:13.851346 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6d4e66f7-c195-4cd7-b842-e12dc53712e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:13.851346 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6d4e66f7-c195-4cd7-b842-e12dc53712e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:13.851346 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6d4e66f7-c195-4cd7-b842-e12dc53712e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:13.851346 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6d4e66f7-c195-4cd7-b842-e12dc53712e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:13.852090 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6d4e66f7-c195-4cd7-b842-e12dc53712e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:13.876658 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6d4e66f7-c195-4cd7-b842-e12dc53712e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Going to try to terminate instance {{(pid=121277) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:32:14.166887 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fa2b1d4a-6f23-41cf-a61d-5cb197e15488 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:14.166887 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fa2b1d4a-6f23-41cf-a61d-5cb197e15488 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:14.173331 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fa2b1d4a-6f23-41cf-a61d-5cb197e15488 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:14.173826 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fa2b1d4a-6f23-41cf-a61d-5cb197e15488 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:14.174272 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fa2b1d4a-6f23-41cf-a61d-5cb197e15488 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:14.174977 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fa2b1d4a-6f23-41cf-a61d-5cb197e15488 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:14.175435 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fa2b1d4a-6f23-41cf-a61d-5cb197e15488 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:14.175872 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fa2b1d4a-6f23-41cf-a61d-5cb197e15488 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:14.214823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6d4e66f7-c195-4cd7-b842-e12dc53712e6 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "DELETE /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 204 len: 0 microversion: 2.1 time: 0.376063 Aug 21 01:32:14.215426 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 437/860] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:32:13 2022] DELETE /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:32:14.224453 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c0e5bbc5-b3d7-4d4f-b4e4-0e91a1a4199f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:14.225167 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c0e5bbc5-b3d7-4d4f-b4e4-0e91a1a4199f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:14.231879 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c0e5bbc5-b3d7-4d4f-b4e4-0e91a1a4199f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:14.232414 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c0e5bbc5-b3d7-4d4f-b4e4-0e91a1a4199f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:14.232861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c0e5bbc5-b3d7-4d4f-b4e4-0e91a1a4199f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:14.233579 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c0e5bbc5-b3d7-4d4f-b4e4-0e91a1a4199f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:14.234042 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c0e5bbc5-b3d7-4d4f-b4e4-0e91a1a4199f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:14.234474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c0e5bbc5-b3d7-4d4f-b4e4-0e91a1a4199f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:14.409350 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fa2b1d4a-6f23-41cf-a61d-5cb197e15488 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:14.418679 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fa2b1d4a-6f23-41cf-a61d-5cb197e15488 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.256248 Aug 21 01:32:14.419279 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 424/861] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:14 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:32:14.434071 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c0e5bbc5-b3d7-4d4f-b4e4-0e91a1a4199f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:14.441902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c0e5bbc5-b3d7-4d4f-b4e4-0e91a1a4199f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1671 microversion: 2.1 time: 0.222165 Aug 21 01:32:14.442465 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 438/862] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:14 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1671 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:32:14.564690 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3d66072c-b9ee-46ba-bb71-eddcff7e241e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:14.565310 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3d66072c-b9ee-46ba-bb71-eddcff7e241e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:14.571297 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3d66072c-b9ee-46ba-bb71-eddcff7e241e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:14.571720 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3d66072c-b9ee-46ba-bb71-eddcff7e241e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:14.572189 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3d66072c-b9ee-46ba-bb71-eddcff7e241e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:14.572881 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3d66072c-b9ee-46ba-bb71-eddcff7e241e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:14.573336 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3d66072c-b9ee-46ba-bb71-eddcff7e241e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:14.573760 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3d66072c-b9ee-46ba-bb71-eddcff7e241e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:14.702437 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3d66072c-b9ee-46ba-bb71-eddcff7e241e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:14.712171 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3d66072c-b9ee-46ba-bb71-eddcff7e241e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:14.713755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3d66072c-b9ee-46ba-bb71-eddcff7e241e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:14.715059 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3d66072c-b9ee-46ba-bb71-eddcff7e241e tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.154261 Aug 21 01:32:14.715726 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 425/863] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:14 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:15.440125 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-691e7ec0-ad22-4627-8783-a53df87418ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:15.440777 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-691e7ec0-ad22-4627-8783-a53df87418ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:15.447887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-691e7ec0-ad22-4627-8783-a53df87418ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:15.448411 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-691e7ec0-ad22-4627-8783-a53df87418ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:15.448847 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-691e7ec0-ad22-4627-8783-a53df87418ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:15.449528 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-691e7ec0-ad22-4627-8783-a53df87418ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:15.449981 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-691e7ec0-ad22-4627-8783-a53df87418ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:15.450402 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-691e7ec0-ad22-4627-8783-a53df87418ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:15.462878 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6662aa71-831c-4ea6-aea9-2e7b7e55af5c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:15.463495 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6662aa71-831c-4ea6-aea9-2e7b7e55af5c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:15.470260 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6662aa71-831c-4ea6-aea9-2e7b7e55af5c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:15.470749 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6662aa71-831c-4ea6-aea9-2e7b7e55af5c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:15.471183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6662aa71-831c-4ea6-aea9-2e7b7e55af5c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:15.471878 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6662aa71-831c-4ea6-aea9-2e7b7e55af5c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:15.472359 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6662aa71-831c-4ea6-aea9-2e7b7e55af5c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:15.472785 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6662aa71-831c-4ea6-aea9-2e7b7e55af5c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:15.655686 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-691e7ec0-ad22-4627-8783-a53df87418ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:15.663433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6662aa71-831c-4ea6-aea9-2e7b7e55af5c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:15.664416 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-691e7ec0-ad22-4627-8783-a53df87418ac tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.228117 Aug 21 01:32:15.665006 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 439/864] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:15 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:15.671510 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6662aa71-831c-4ea6-aea9-2e7b7e55af5c tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1671 microversion: 2.1 time: 0.212467 Aug 21 01:32:15.672078 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 426/865] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:15 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1671 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:32:15.739451 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8ae9fdd4-44a4-4da9-b792-c0a3c975fe8b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:15.740058 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8ae9fdd4-44a4-4da9-b792-c0a3c975fe8b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:15.747137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8ae9fdd4-44a4-4da9-b792-c0a3c975fe8b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:15.747617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8ae9fdd4-44a4-4da9-b792-c0a3c975fe8b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:15.748051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8ae9fdd4-44a4-4da9-b792-c0a3c975fe8b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:15.748759 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8ae9fdd4-44a4-4da9-b792-c0a3c975fe8b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:15.749244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8ae9fdd4-44a4-4da9-b792-c0a3c975fe8b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:15.749624 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8ae9fdd4-44a4-4da9-b792-c0a3c975fe8b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:15.887397 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8ae9fdd4-44a4-4da9-b792-c0a3c975fe8b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:15.895579 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8ae9fdd4-44a4-4da9-b792-c0a3c975fe8b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:15.897115 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8ae9fdd4-44a4-4da9-b792-c0a3c975fe8b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:15.898296 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8ae9fdd4-44a4-4da9-b792-c0a3c975fe8b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.163400 Aug 21 01:32:15.898849 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 440/866] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:15 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:16.685556 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4d69f91b-5f71-49fa-a3af-ea0cf1540c3a tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:16.687845 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4d69f91b-5f71-49fa-a3af-ea0cf1540c3a tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:16.693518 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4d69f91b-5f71-49fa-a3af-ea0cf1540c3a tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:16.694010 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4d69f91b-5f71-49fa-a3af-ea0cf1540c3a tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:16.694448 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4d69f91b-5f71-49fa-a3af-ea0cf1540c3a tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:16.695147 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4d69f91b-5f71-49fa-a3af-ea0cf1540c3a tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:16.695264 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a299dc2e-865f-487a-afc3-35c67aacb209 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:16.695768 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4d69f91b-5f71-49fa-a3af-ea0cf1540c3a tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:16.695884 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a299dc2e-865f-487a-afc3-35c67aacb209 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:16.696001 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4d69f91b-5f71-49fa-a3af-ea0cf1540c3a tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:16.702687 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a299dc2e-865f-487a-afc3-35c67aacb209 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:16.703243 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a299dc2e-865f-487a-afc3-35c67aacb209 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:16.704228 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a299dc2e-865f-487a-afc3-35c67aacb209 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:16.704730 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a299dc2e-865f-487a-afc3-35c67aacb209 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:16.705180 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a299dc2e-865f-487a-afc3-35c67aacb209 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:16.705604 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a299dc2e-865f-487a-afc3-35c67aacb209 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:17.058036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4d69f91b-5f71-49fa-a3af-ea0cf1540c3a tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:17.064746 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4d69f91b-5f71-49fa-a3af-ea0cf1540c3a tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.382894 Aug 21 01:32:17.074914 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 427/867] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:16 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:17.074914 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e428e341-5abe-411e-807e-488e4a72b96a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:17.074914 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e428e341-5abe-411e-807e-488e4a72b96a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:17.080306 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e428e341-5abe-411e-807e-488e4a72b96a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:17.081091 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e428e341-5abe-411e-807e-488e4a72b96a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:17.081737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e428e341-5abe-411e-807e-488e4a72b96a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:17.082682 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e428e341-5abe-411e-807e-488e4a72b96a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:17.083218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e428e341-5abe-411e-807e-488e4a72b96a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:17.083701 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e428e341-5abe-411e-807e-488e4a72b96a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:17.107111 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a299dc2e-865f-487a-afc3-35c67aacb209 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:17.120749 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a299dc2e-865f-487a-afc3-35c67aacb209 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1671 microversion: 2.1 time: 0.431046 Aug 21 01:32:17.132259 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 441/868] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:16 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1671 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:32:17.372656 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e428e341-5abe-411e-807e-488e4a72b96a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:17.381421 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e428e341-5abe-411e-807e-488e4a72b96a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:17.383058 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e428e341-5abe-411e-807e-488e4a72b96a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:17.384256 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e428e341-5abe-411e-807e-488e4a72b96a tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1727 microversion: 2.20 time: 0.317370 Aug 21 01:32:17.384816 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 428/869] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:17 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1727 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:18.090204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c07c7deb-1efc-40bf-a653-1d38860147a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:18.091056 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c07c7deb-1efc-40bf-a653-1d38860147a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:18.098497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c07c7deb-1efc-40bf-a653-1d38860147a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:18.099094 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c07c7deb-1efc-40bf-a653-1d38860147a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:18.099542 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c07c7deb-1efc-40bf-a653-1d38860147a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:18.100286 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c07c7deb-1efc-40bf-a653-1d38860147a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:18.100759 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c07c7deb-1efc-40bf-a653-1d38860147a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:18.101189 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c07c7deb-1efc-40bf-a653-1d38860147a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:18.146570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fb824c77-22e8-4db7-9eef-da91a7c7fcd7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:18.147268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fb824c77-22e8-4db7-9eef-da91a7c7fcd7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:18.154196 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fb824c77-22e8-4db7-9eef-da91a7c7fcd7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:18.155020 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fb824c77-22e8-4db7-9eef-da91a7c7fcd7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:18.155462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fb824c77-22e8-4db7-9eef-da91a7c7fcd7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:18.156227 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fb824c77-22e8-4db7-9eef-da91a7c7fcd7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:18.156650 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fb824c77-22e8-4db7-9eef-da91a7c7fcd7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:18.157130 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fb824c77-22e8-4db7-9eef-da91a7c7fcd7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:18.285021 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fb824c77-22e8-4db7-9eef-da91a7c7fcd7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:18.293196 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fb824c77-22e8-4db7-9eef-da91a7c7fcd7 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1606 microversion: 2.1 time: 0.151246 Aug 21 01:32:18.293786 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 429/870] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:18 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1606 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:32:18.407458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-203254ee-7f63-4f33-b323-858aab67cd1b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:18.408270 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-203254ee-7f63-4f33-b323-858aab67cd1b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:18.414441 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-203254ee-7f63-4f33-b323-858aab67cd1b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:18.414940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-203254ee-7f63-4f33-b323-858aab67cd1b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:18.415383 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-203254ee-7f63-4f33-b323-858aab67cd1b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:18.416114 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-203254ee-7f63-4f33-b323-858aab67cd1b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:18.416582 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-203254ee-7f63-4f33-b323-858aab67cd1b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:18.417029 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-203254ee-7f63-4f33-b323-858aab67cd1b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:18.462167 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c07c7deb-1efc-40bf-a653-1d38860147a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:18.464823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:18.471251 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c07c7deb-1efc-40bf-a653-1d38860147a8 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.384876 Aug 21 01:32:18.471838 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 442/871] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:18 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:18.772482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-203254ee-7f63-4f33-b323-858aab67cd1b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:18.781095 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-203254ee-7f63-4f33-b323-858aab67cd1b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:18.783810 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-203254ee-7f63-4f33-b323-858aab67cd1b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:18.783935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-203254ee-7f63-4f33-b323-858aab67cd1b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1714 microversion: 2.20 time: 0.380442 Aug 21 01:32:18.784534 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 430/872] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:18 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1714 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:32:18.872497 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-6619ea36-84e6-41ea-9fdb-d43a16978e23 req-0983f62c-85a9-44ce-9b9e-b1f5627869c2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a075e00-151c-4ac4-b454-deea3de38ef7", "name": "network-vif-deleted", "tag": "306889a6-f7da-46ac-91cc-f0f0db7e57c3"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:32:18.883577 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-6619ea36-84e6-41ea-9fdb-d43a16978e23 req-0983f62c-85a9-44ce-9b9e-b1f5627869c2 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:18.884178 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-6619ea36-84e6-41ea-9fdb-d43a16978e23 req-0983f62c-85a9-44ce-9b9e-b1f5627869c2 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:18.884688 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-6619ea36-84e6-41ea-9fdb-d43a16978e23 req-0983f62c-85a9-44ce-9b9e-b1f5627869c2 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:18.895991 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-6619ea36-84e6-41ea-9fdb-d43a16978e23 req-0983f62c-85a9-44ce-9b9e-b1f5627869c2 service nova] Creating event network-vif-deleted:306889a6-f7da-46ac-91cc-f0f0db7e57c3 for instance 9a075e00-151c-4ac4-b454-deea3de38ef7 on hv-cinder-maste Aug 21 01:32:18.901516 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-6619ea36-84e6-41ea-9fdb-d43a16978e23 req-0983f62c-85a9-44ce-9b9e-b1f5627869c2 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033656 Aug 21 01:32:18.902193 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 443/873] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:32:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:32:19.314036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:19.314677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:19.322114 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:19.323338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:19.323815 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:19.324735 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:19.325205 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:19.325642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:19.361464 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:19.361954 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:19.362398 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:19.362854 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lazy-loading 'fault' on Instance uuid 9a075e00-151c-4ac4-b454-deea3de38ef7 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:32:19.405523 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0cd8d822f15943a78575c3744692cdb9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4a0fcea08b248fe84012875a7243761', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:19.413965 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-810634fe-6715-4030-8a2b-8e82ec7e5b9d tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 200 len: 1327 microversion: 2.1 time: 0.103805 Aug 21 01:32:19.414550 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 431/874] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:19 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 1327 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:32:19.496034 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7e9d7036-59e8-4309-8098-0a20735c3396 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:19.497140 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7e9d7036-59e8-4309-8098-0a20735c3396 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:19.505918 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7e9d7036-59e8-4309-8098-0a20735c3396 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:19.506521 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7e9d7036-59e8-4309-8098-0a20735c3396 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:19.507059 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7e9d7036-59e8-4309-8098-0a20735c3396 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:19.507964 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7e9d7036-59e8-4309-8098-0a20735c3396 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:19.508618 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7e9d7036-59e8-4309-8098-0a20735c3396 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:19.518720 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7e9d7036-59e8-4309-8098-0a20735c3396 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:19.668837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7e9d7036-59e8-4309-8098-0a20735c3396 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:19.679480 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7e9d7036-59e8-4309-8098-0a20735c3396 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.189365 Aug 21 01:32:19.685626 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 444/875] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:19 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:20.394243 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2c28fe2d-f906-4d94-898a-b123069822f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:20.394833 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2c28fe2d-f906-4d94-898a-b123069822f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:20.401944 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c28fe2d-f906-4d94-898a-b123069822f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:20.402430 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c28fe2d-f906-4d94-898a-b123069822f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:20.402863 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c28fe2d-f906-4d94-898a-b123069822f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:20.403561 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c28fe2d-f906-4d94-898a-b123069822f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:20.404007 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c28fe2d-f906-4d94-898a-b123069822f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:20.404458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c28fe2d-f906-4d94-898a-b123069822f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:20.428643 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2c28fe2d-f906-4d94-898a-b123069822f3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/ips" status: 200 len: 149 microversion: 2.1 time: 0.038165 Aug 21 01:32:20.429190 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 432/876] 10.222.0.47 () {58 vars in 1248 bytes} [Sun Aug 21 01:32:20 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/ips => generated 149 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:32:20.440850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7c100046-adc4-4642-a942-cc571c179835 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:20.441488 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7c100046-adc4-4642-a942-cc571c179835 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:20.448262 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7c100046-adc4-4642-a942-cc571c179835 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:20.448757 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7c100046-adc4-4642-a942-cc571c179835 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:20.449951 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7c100046-adc4-4642-a942-cc571c179835 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:20.449951 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7c100046-adc4-4642-a942-cc571c179835 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:20.450583 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7c100046-adc4-4642-a942-cc571c179835 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:20.450893 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7c100046-adc4-4642-a942-cc571c179835 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:20.465490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-7c100046-adc4-4642-a942-cc571c179835 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] HTTP exception thrown: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7c100046-adc4-4642-a942-cc571c179835 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Request method failure captured: Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 HTTP/1.1 Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYl7pq5WVSQzS51LpLVCZZUw27iHEeVZJoQRcIvRcn8IboU95JITc9pVqybD2xzZX2nmxh3OfTWCk6tel2UXiN18o1XKAXaRJqQ1AV1sQ2sNj8ZWXwdNkHZXM2QMX9i5RlP6rG_R721C3RJRTJbjQ-MvmqR7-hIA4GefX0KAYXbHx7g Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: f4a0fcea08b248fe84012875a7243761 Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:32:20.467146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:32:20.468875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_f4a0fcea08b248fe84012875a7243761"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:32:20.468875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:20.468875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: f4a0fcea08b248fe84012875a7243761 Aug 21 01:32:20.468875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:20.468875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeTestJSON-528571573-project Aug 21 01:32:20.468875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:32:20.468875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:32:20.468875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 0cd8d822f15943a78575c3744692cdb9 Aug 21 01:32:20.468875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeTestJSON-528571573-project Aug 21 01:32:20.468875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:32:20.468875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:20.468875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '9a075e00-151c-4ac4-b454-deea3de38ef7'} Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:32:20.469745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:20.471053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:20.472315 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7c100046-adc4-4642-a942-cc571c179835 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Returning 404 to user: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:32:20.472315 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7c100046-adc4-4642-a942-cc571c179835 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 404 len: 111 microversion: 2.1 time: 0.033241 Aug 21 01:32:20.472315 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 445/877] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:20 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:32:20.477854 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-210c6fdd-8a4a-455d-b8e7-7a9ddf62bc0e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '.version_select at 0x7f9da3375700>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:20.562778 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-210c6fdd-8a4a-455d-b8e7-7a9ddf62bc0e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-899759034" status: 202 len: 0 microversion: 2.1 time: 0.088599 Aug 21 01:32:20.563388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 433/878] 10.222.0.47 () {58 vars in 1218 bytes} [Sun Aug 21 01:32:20 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-899759034 => generated 0 bytes in 90 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:32:20.700923 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5a5ee7d2-1577-43d3-bf0a-1d6053128cce tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:20.701632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5a5ee7d2-1577-43d3-bf0a-1d6053128cce tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:20.708832 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a5ee7d2-1577-43d3-bf0a-1d6053128cce tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:20.714995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a5ee7d2-1577-43d3-bf0a-1d6053128cce tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:20.714995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a5ee7d2-1577-43d3-bf0a-1d6053128cce tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:20.714995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a5ee7d2-1577-43d3-bf0a-1d6053128cce tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:20.714995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a5ee7d2-1577-43d3-bf0a-1d6053128cce tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:20.715592 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5a5ee7d2-1577-43d3-bf0a-1d6053128cce tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:20.846090 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5a5ee7d2-1577-43d3-bf0a-1d6053128cce tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:20.855064 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5a5ee7d2-1577-43d3-bf0a-1d6053128cce tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.157945 Aug 21 01:32:20.856189 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 446/879] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:20 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:21.877886 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-30ef6467-d243-411b-8fb5-afc95aa988d1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:21.878503 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-30ef6467-d243-411b-8fb5-afc95aa988d1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:21.885372 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30ef6467-d243-411b-8fb5-afc95aa988d1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:21.885856 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30ef6467-d243-411b-8fb5-afc95aa988d1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:21.893939 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30ef6467-d243-411b-8fb5-afc95aa988d1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:21.893939 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30ef6467-d243-411b-8fb5-afc95aa988d1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:21.893939 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30ef6467-d243-411b-8fb5-afc95aa988d1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:21.893939 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-30ef6467-d243-411b-8fb5-afc95aa988d1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:21.894330 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=100 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:22.033645 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-30ef6467-d243-411b-8fb5-afc95aa988d1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:22.037757 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-30ef6467-d243-411b-8fb5-afc95aa988d1 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.164262 Aug 21 01:32:22.038527 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 434/880] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:21 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:22.352378 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4f1ef17c-060d-4b52-8dbc-67d3192a243b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "fc6ceacd-e32a-4de3-bd34-144c0e60919d"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:32:22.354797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4f1ef17c-060d-4b52-8dbc-67d3192a243b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:22.361257 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f1ef17c-060d-4b52-8dbc-67d3192a243b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:22.361736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f1ef17c-060d-4b52-8dbc-67d3192a243b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:22.362168 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f1ef17c-060d-4b52-8dbc-67d3192a243b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:22.362859 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f1ef17c-060d-4b52-8dbc-67d3192a243b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:22.363314 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f1ef17c-060d-4b52-8dbc-67d3192a243b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:22.363735 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f1ef17c-060d-4b52-8dbc-67d3192a243b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:23.059374 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-012ab8ff-dc24-4be0-ac57-60a25b649d2f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:23.059907 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-012ab8ff-dc24-4be0-ac57-60a25b649d2f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:23.066863 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-012ab8ff-dc24-4be0-ac57-60a25b649d2f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:23.067430 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-012ab8ff-dc24-4be0-ac57-60a25b649d2f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:23.068051 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-012ab8ff-dc24-4be0-ac57-60a25b649d2f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:23.068937 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-012ab8ff-dc24-4be0-ac57-60a25b649d2f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:23.070423 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-012ab8ff-dc24-4be0-ac57-60a25b649d2f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:23.071931 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-012ab8ff-dc24-4be0-ac57-60a25b649d2f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:23.214159 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-012ab8ff-dc24-4be0-ac57-60a25b649d2f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:23.222489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-012ab8ff-dc24-4be0-ac57-60a25b649d2f tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.167009 Aug 21 01:32:23.223066 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 435/881] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:23 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:32:23.372194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4f1ef17c-060d-4b52-8dbc-67d3192a243b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.024157 Aug 21 01:32:23.372793 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 447/882] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:32:22 2022] POST /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/os-volume_attachments => generated 194 bytes in 1026 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:32:24.245406 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-819d9cf4-2237-435a-b0a7-c8b24d793096 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:24.246084 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-819d9cf4-2237-435a-b0a7-c8b24d793096 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:24.252698 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-819d9cf4-2237-435a-b0a7-c8b24d793096 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:24.253184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-819d9cf4-2237-435a-b0a7-c8b24d793096 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:24.253992 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-819d9cf4-2237-435a-b0a7-c8b24d793096 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:24.254306 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-819d9cf4-2237-435a-b0a7-c8b24d793096 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:24.254756 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-819d9cf4-2237-435a-b0a7-c8b24d793096 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:24.255180 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-819d9cf4-2237-435a-b0a7-c8b24d793096 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:24.282155 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5469abe4-d75e-41fe-b1f3-07599d41d50e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:24.282740 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5469abe4-d75e-41fe-b1f3-07599d41d50e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:24.284693 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=75,nova_api:UPDATE=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:24.289890 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5469abe4-d75e-41fe-b1f3-07599d41d50e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:24.290410 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5469abe4-d75e-41fe-b1f3-07599d41d50e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:24.290848 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5469abe4-d75e-41fe-b1f3-07599d41d50e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:24.291536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5469abe4-d75e-41fe-b1f3-07599d41d50e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:24.292222 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5469abe4-d75e-41fe-b1f3-07599d41d50e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:24.292531 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5469abe4-d75e-41fe-b1f3-07599d41d50e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:24.307548 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-5469abe4-d75e-41fe-b1f3-07599d41d50e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] HTTP exception thrown: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5469abe4-d75e-41fe-b1f3-07599d41d50e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Request method failure captured: Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: DELETE /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 HTTP/1.1 Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYl7pq5WVSQzS51LpLVCZZUw27iHEeVZJoQRcIvRcn8IboU95JITc9pVqybD2xzZX2nmxh3OfTWCk6tel2UXiN18o1XKAXaRJqQ1AV1sQ2sNj8ZWXwdNkHZXM2QMX9i5RlP6rG_R721C3RJRTJbjQ-MvmqR7-hIA4GefX0KAYXbHx7g Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: f4a0fcea08b248fe84012875a7243761 Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:32:24.309372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:32:24.310946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_f4a0fcea08b248fe84012875a7243761"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:32:24.310946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:24.310946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: f4a0fcea08b248fe84012875a7243761 Aug 21 01:32:24.310946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:24.310946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeTestJSON-528571573-project Aug 21 01:32:24.310946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:32:24.310946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:32:24.310946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 0cd8d822f15943a78575c3744692cdb9 Aug 21 01:32:24.310946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeTestJSON-528571573-project Aug 21 01:32:24.310946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:32:24.310946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:24.310946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '9a075e00-151c-4ac4-b454-deea3de38ef7'} Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:32:24.311795 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:24.313282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:24.315113 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5469abe4-d75e-41fe-b1f3-07599d41d50e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Returning 404 to user: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:32:24.315113 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5469abe4-d75e-41fe-b1f3-07599d41d50e tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "DELETE /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 404 len: 111 microversion: 2.1 time: 0.033672 Aug 21 01:32:24.315113 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 448/883] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:32:24 2022] DELETE /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 21 01:32:24.319268 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f7792858-a0fa-45d7-a547-61e71a51e238 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:24.320536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f7792858-a0fa-45d7-a547-61e71a51e238 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: 9a075e00-151c-4ac4-b454-deea3de38ef7] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:24.326822 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7792858-a0fa-45d7-a547-61e71a51e238 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:24.327405 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7792858-a0fa-45d7-a547-61e71a51e238 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:24.332592 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7792858-a0fa-45d7-a547-61e71a51e238 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:24.332592 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7792858-a0fa-45d7-a547-61e71a51e238 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:24.332592 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7792858-a0fa-45d7-a547-61e71a51e238 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:24.332592 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f7792858-a0fa-45d7-a547-61e71a51e238 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:24.347038 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-f7792858-a0fa-45d7-a547-61e71a51e238 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] HTTP exception thrown: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f7792858-a0fa-45d7-a547-61e71a51e238 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Request method failure captured: Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 HTTP/1.1 Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYl7pq5WVSQzS51LpLVCZZUw27iHEeVZJoQRcIvRcn8IboU95JITc9pVqybD2xzZX2nmxh3OfTWCk6tel2UXiN18o1XKAXaRJqQ1AV1sQ2sNj8ZWXwdNkHZXM2QMX9i5RlP6rG_R721C3RJRTJbjQ-MvmqR7-hIA4GefX0KAYXbHx7g Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: f4a0fcea08b248fe84012875a7243761 Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:32:24.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:32:24.350234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_f4a0fcea08b248fe84012875a7243761"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:32:24.350234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:24.350234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: f4a0fcea08b248fe84012875a7243761 Aug 21 01:32:24.350234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:24.350234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeTestJSON-528571573-project Aug 21 01:32:24.350234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:32:24.350234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:32:24.350234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 0cd8d822f15943a78575c3744692cdb9 Aug 21 01:32:24.350234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeTestJSON-528571573-project Aug 21 01:32:24.350234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:32:24.350234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:24.350234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '9a075e00-151c-4ac4-b454-deea3de38ef7'} Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:32:24.351137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. Aug 21 01:32:24.353206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:24.354708 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f7792858-a0fa-45d7-a547-61e71a51e238 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Returning 404 to user: Instance 9a075e00-151c-4ac4-b454-deea3de38ef7 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:32:24.354708 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f7792858-a0fa-45d7-a547-61e71a51e238 tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7" status: 404 len: 111 microversion: 2.1 time: 0.034553 Aug 21 01:32:24.354708 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 449/884] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:24 2022] GET /compute/v2.1/servers/9a075e00-151c-4ac4-b454-deea3de38ef7 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 21 01:32:24.406087 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-819d9cf4-2237-435a-b0a7-c8b24d793096 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:24.413851 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-819d9cf4-2237-435a-b0a7-c8b24d793096 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.172771 Aug 21 01:32:24.414417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 436/885] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:24 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:25.435245 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-dc598882-0284-45ef-96b7-e53ea5920d05 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:25.435919 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-dc598882-0284-45ef-96b7-e53ea5920d05 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:25.442727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dc598882-0284-45ef-96b7-e53ea5920d05 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:25.443217 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dc598882-0284-45ef-96b7-e53ea5920d05 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:25.443653 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dc598882-0284-45ef-96b7-e53ea5920d05 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:25.444899 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dc598882-0284-45ef-96b7-e53ea5920d05 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:25.445357 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dc598882-0284-45ef-96b7-e53ea5920d05 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:25.445779 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-dc598882-0284-45ef-96b7-e53ea5920d05 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:25.582846 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-dc598882-0284-45ef-96b7-e53ea5920d05 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:25.591687 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-dc598882-0284-45ef-96b7-e53ea5920d05 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.160541 Aug 21 01:32:25.596968 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 450/886] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:25 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:25.747366 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-18bf86e6-36f2-4ad0-bd6e-3c44c04e23fc tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:25.747871 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-18bf86e6-36f2-4ad0-bd6e-3c44c04e23fc tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:25.754394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-18bf86e6-36f2-4ad0-bd6e-3c44c04e23fc tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:25.754876 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-18bf86e6-36f2-4ad0-bd6e-3c44c04e23fc tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:25.755314 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-18bf86e6-36f2-4ad0-bd6e-3c44c04e23fc tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:25.756008 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-18bf86e6-36f2-4ad0-bd6e-3c44c04e23fc tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:25.756495 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-18bf86e6-36f2-4ad0-bd6e-3c44c04e23fc tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:25.757582 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-18bf86e6-36f2-4ad0-bd6e-3c44c04e23fc tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:25.770869 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-18bf86e6-36f2-4ad0-bd6e-3c44c04e23fc tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] HTTP exception thrown: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-18bf86e6-36f2-4ad0-bd6e-3c44c04e23fc tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Request method failure captured: Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: DELETE /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 HTTP/1.1 Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYl7pq5WVSQzS51LpLVCZZUw27iHEeVZJoQRcIvRcn8IboU95JITc9pVqybD2xzZX2nmxh3OfTWCk6tel2UXiN18o1XKAXaRJqQ1AV1sQ2sNj8ZWXwdNkHZXM2QMX9i5RlP6rG_R721C3RJRTJbjQ-MvmqR7-hIA4GefX0KAYXbHx7g Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: f4a0fcea08b248fe84012875a7243761 Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:32:25.772562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:32:25.774150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_f4a0fcea08b248fe84012875a7243761"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:32:25.774150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:25.774150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: f4a0fcea08b248fe84012875a7243761 Aug 21 01:32:25.774150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:25.774150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeTestJSON-528571573-project Aug 21 01:32:25.774150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:32:25.774150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:32:25.774150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: 0cd8d822f15943a78575c3744692cdb9 Aug 21 01:32:25.774150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeTestJSON-528571573-project Aug 21 01:32:25.774150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:32:25.774150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:32:25.774150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': 'c266a09a-3f81-4111-a987-b1f4be65cdc4'} Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:32:25.774933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:32:25.776121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:25.777323 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-18bf86e6-36f2-4ad0-bd6e-3c44c04e23fc tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Returning 404 to user: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:32:25.777323 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-18bf86e6-36f2-4ad0-bd6e-3c44c04e23fc tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "DELETE /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 404 len: 111 microversion: 2.1 time: 0.030381 Aug 21 01:32:25.777323 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 437/887] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:32:25 2022] DELETE /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:32:25.782646 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-673e3ab8-bc3b-42ee-a5a5-9d97ea58ac9f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:25.783267 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-673e3ab8-bc3b-42ee-a5a5-9d97ea58ac9f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] [instance: c266a09a-3f81-4111-a987-b1f4be65cdc4] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:25.789206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-673e3ab8-bc3b-42ee-a5a5-9d97ea58ac9f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:25.789695 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-673e3ab8-bc3b-42ee-a5a5-9d97ea58ac9f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:25.790239 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-673e3ab8-bc3b-42ee-a5a5-9d97ea58ac9f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:25.790935 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-673e3ab8-bc3b-42ee-a5a5-9d97ea58ac9f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:25.791568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-673e3ab8-bc3b-42ee-a5a5-9d97ea58ac9f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:25.791918 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-673e3ab8-bc3b-42ee-a5a5-9d97ea58ac9f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:25.805768 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-673e3ab8-bc3b-42ee-a5a5-9d97ea58ac9f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] HTTP exception thrown: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-673e3ab8-bc3b-42ee-a5a5-9d97ea58ac9f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Request method failure captured: Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 HTTP/1.1 Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYl7pq5WVSQzS51LpLVCZZUw27iHEeVZJoQRcIvRcn8IboU95JITc9pVqybD2xzZX2nmxh3OfTWCk6tel2UXiN18o1XKAXaRJqQ1AV1sQ2sNj8ZWXwdNkHZXM2QMX9i5RlP6rG_R721C3RJRTJbjQ-MvmqR7-hIA4GefX0KAYXbHx7g Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: f4a0fcea08b248fe84012875a7243761 Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:32:25.807297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:32:25.808933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_f4a0fcea08b248fe84012875a7243761"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/f4a0fcea08b248fe84012875a7243761"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:32:25.808933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:25.808933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: f4a0fcea08b248fe84012875a7243761 Aug 21 01:32:25.808933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeTestJSON-528571573 Aug 21 01:32:25.808933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeTestJSON-528571573-project Aug 21 01:32:25.808933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:32:25.808933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:32:25.808933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 0cd8d822f15943a78575c3744692cdb9 Aug 21 01:32:25.808933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeTestJSON-528571573-project Aug 21 01:32:25.808933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:32:25.808933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:32:25.808933 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': 'c266a09a-3f81-4111-a987-b1f4be65cdc4'} Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:32:25.809902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. Aug 21 01:32:25.811702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:25.816642 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-673e3ab8-bc3b-42ee-a5a5-9d97ea58ac9f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] Returning 404 to user: Instance c266a09a-3f81-4111-a987-b1f4be65cdc4 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:32:25.816642 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-673e3ab8-bc3b-42ee-a5a5-9d97ea58ac9f tempest-AttachVolumeTestJSON-528571573 tempest-AttachVolumeTestJSON-528571573-project] 10.222.0.47 "GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4" status: 404 len: 111 microversion: 2.1 time: 0.030014 Aug 21 01:32:25.816642 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 451/888] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:25 2022] GET /compute/v2.1/servers/c266a09a-3f81-4111-a987-b1f4be65cdc4 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:32:26.613087 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f54b3776-bddf-45dc-bcd7-b66832ff1ecc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:26.613621 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f54b3776-bddf-45dc-bcd7-b66832ff1ecc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:26.619697 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f54b3776-bddf-45dc-bcd7-b66832ff1ecc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:26.620212 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f54b3776-bddf-45dc-bcd7-b66832ff1ecc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:26.620651 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f54b3776-bddf-45dc-bcd7-b66832ff1ecc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:26.621343 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f54b3776-bddf-45dc-bcd7-b66832ff1ecc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:26.621797 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f54b3776-bddf-45dc-bcd7-b66832ff1ecc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:26.622249 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f54b3776-bddf-45dc-bcd7-b66832ff1ecc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:26.914241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f54b3776-bddf-45dc-bcd7-b66832ff1ecc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:26.923540 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f54b3776-bddf-45dc-bcd7-b66832ff1ecc tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.314301 Aug 21 01:32:26.924409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 438/889] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:26 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:32:27.815679 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-58f1d784-d13e-4ccf-8a7e-dea15dcc83b6 req-307ebb64-300d-45a1-948c-d46efbcdb54d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "119204ec-e0f8-439e-b985-b934d251684f", "name": "network-vif-plugged", "status": "completed", "tag": "bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:32:27.830738 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-58f1d784-d13e-4ccf-8a7e-dea15dcc83b6 req-307ebb64-300d-45a1-948c-d46efbcdb54d service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:27.831225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-58f1d784-d13e-4ccf-8a7e-dea15dcc83b6 req-307ebb64-300d-45a1-948c-d46efbcdb54d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:27.831712 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-58f1d784-d13e-4ccf-8a7e-dea15dcc83b6 req-307ebb64-300d-45a1-948c-d46efbcdb54d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:27.844943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-58f1d784-d13e-4ccf-8a7e-dea15dcc83b6 req-307ebb64-300d-45a1-948c-d46efbcdb54d service nova] Creating event network-vif-plugged:bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f for instance 119204ec-e0f8-439e-b985-b934d251684f on hv-cinder-maste Aug 21 01:32:27.850756 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-58f1d784-d13e-4ccf-8a7e-dea15dcc83b6 req-307ebb64-300d-45a1-948c-d46efbcdb54d service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039520 Aug 21 01:32:27.851911 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 452/890] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:32:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:32:27.946588 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e880b77b-5c1f-42be-ab23-85aa919efdbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:27.947176 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e880b77b-5c1f-42be-ab23-85aa919efdbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:27.953714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e880b77b-5c1f-42be-ab23-85aa919efdbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:27.954731 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e880b77b-5c1f-42be-ab23-85aa919efdbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:27.955200 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e880b77b-5c1f-42be-ab23-85aa919efdbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:27.955909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e880b77b-5c1f-42be-ab23-85aa919efdbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:27.956404 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e880b77b-5c1f-42be-ab23-85aa919efdbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:27.956826 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e880b77b-5c1f-42be-ab23-85aa919efdbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:28.103663 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e880b77b-5c1f-42be-ab23-85aa919efdbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:28.107375 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e880b77b-5c1f-42be-ab23-85aa919efdbb tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.165374 Aug 21 01:32:28.107957 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 439/891] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:27 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:29.131448 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d8b6dd0c-f16b-4efa-b7f8-e94646757ef4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:29.132084 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d8b6dd0c-f16b-4efa-b7f8-e94646757ef4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:29.143743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d8b6dd0c-f16b-4efa-b7f8-e94646757ef4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:29.144640 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d8b6dd0c-f16b-4efa-b7f8-e94646757ef4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:29.145267 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d8b6dd0c-f16b-4efa-b7f8-e94646757ef4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:29.146281 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d8b6dd0c-f16b-4efa-b7f8-e94646757ef4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:29.146962 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d8b6dd0c-f16b-4efa-b7f8-e94646757ef4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:29.147576 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d8b6dd0c-f16b-4efa-b7f8-e94646757ef4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:29.307332 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d8b6dd0c-f16b-4efa-b7f8-e94646757ef4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:29.316039 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d8b6dd0c-f16b-4efa-b7f8-e94646757ef4 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1575 microversion: 2.1 time: 0.189383 Aug 21 01:32:29.317200 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 453/892] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:29 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1575 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:30.339759 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d6bebe9d-666f-45f3-861d-7794712966c0 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:30.340499 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d6bebe9d-666f-45f3-861d-7794712966c0 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:30.358802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6bebe9d-666f-45f3-861d-7794712966c0 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:30.358802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6bebe9d-666f-45f3-861d-7794712966c0 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:30.358802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6bebe9d-666f-45f3-861d-7794712966c0 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:30.358802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6bebe9d-666f-45f3-861d-7794712966c0 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:30.359176 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6bebe9d-666f-45f3-861d-7794712966c0 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:30.359176 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d6bebe9d-666f-45f3-861d-7794712966c0 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:30.508531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d6bebe9d-666f-45f3-861d-7794712966c0 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:30.516405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d6bebe9d-666f-45f3-861d-7794712966c0 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1562 microversion: 2.1 time: 0.181068 Aug 21 01:32:30.516965 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 440/893] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:30 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1562 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:31.520668 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5a83a81b-3421-4463-9c3d-a8e0d580f058 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] Action: 'create', calling method: .version_select at 0x7f9da34ea820>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-757207564", "metadata": {"Type": "work"}}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:32:32.206428 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ef97f49f-3446-4512-b3e2-ba74dc161a19 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: .version_select at 0x7f9da337e5e0>, body: {"keypair": {"name": "tempest-keypair-1210986866", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHvFtf4dNudKLUeY/TZcD93WO7VS4ddeV/KnxpIHlRZcqoTKJz/QzZh6hK173wQlHONXtmvHmE6xFlXHh72bsqyYCHLKeJQQOi85EJ0ozVklCt7QpPmGgZBd4/VmPULN3w=="}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:32:32.222421 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-ef97f49f-3446-4512-b3e2-ba74dc161a19 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for project 122b46d9e29843a88c7ca5f019426539. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:32:32.233931 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-ef97f49f-3446-4512-b3e2-ba74dc161a19 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for user a91ac0ba7e8c4418930852b0fa184247 and project 122b46d9e29843a88c7ca5f019426539. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:32:32.281691 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5a83a81b-3421-4463-9c3d-a8e0d580f058 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] 10.222.0.47 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 0.874115 Aug 21 01:32:32.282354 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 454/894] 10.222.0.47 () {60 vars in 1156 bytes} [Sun Aug 21 01:32:31 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 875 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Aug 21 01:32:32.294883 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7c775ae5-b960-4fc9-98c1-073319f404c8 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] Calling method '.version_select at 0x7f9da3263af0>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:32.342242 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7c775ae5-b960-4fc9-98c1-073319f404c8 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031" status: 200 len: 352 microversion: 2.1 time: 0.050930 Aug 21 01:32:32.342806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 455/895] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:32:32 2022] GET /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031 => generated 352 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:32:32.388965 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ef97f49f-3446-4512-b3e2-ba74dc161a19 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.186503 Aug 21 01:32:32.390085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 441/896] 10.222.0.47 () {60 vars in 1159 bytes} [Sun Aug 21 01:32:32 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:32:33.355844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-bc5e15b4-b862-4609-a04e-148bfd7e4644 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] Calling method '.version_select at 0x7f9da3393820>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:33.403952 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-bc5e15b4-b862-4609-a04e-148bfd7e4644 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031" status: 200 len: 353 microversion: 2.1 time: 0.052404 Aug 21 01:32:33.404553 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 456/897] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:32:33 2022] GET /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031 => generated 353 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:32:33.416954 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ca498145-c420-4745-8770-0bd72c3d689c tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] Calling method '.version_select at 0x7f9da3365670>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:33.464823 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ca498145-c420-4745-8770-0bd72c3d689c tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031" status: 200 len: 353 microversion: 2.1 time: 0.051887 Aug 21 01:32:33.465386 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 442/898] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:32:33 2022] GET /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031 => generated 353 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:32:33.482662 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-682f52a2-b200-4954-9b58-0faba090c5cf tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] Calling method '.version_select at 0x7f9da32635e0>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:33.769102 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-682f52a2-b200-4954-9b58-0faba090c5cf tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] 10.222.0.47 "DELETE /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031" status: 202 len: 0 microversion: 2.1 time: 0.290145 Aug 21 01:32:33.769668 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 457/899] 10.222.0.47 () {58 vars in 1248 bytes} [Sun Aug 21 01:32:33 2022] DELETE /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031 => generated 0 bytes in 292 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:32:33.777821 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7738042e-6355-4b24-ad2b-35832e64c857 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] Calling method '.version_select at 0x7f9da33851f0>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:33.825916 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7738042e-6355-4b24-ad2b-35832e64c857 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031" status: 200 len: 352 microversion: 2.1 time: 0.052007 Aug 21 01:32:33.826522 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 443/900] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:32:33 2022] GET /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031 => generated 352 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:32:34.840521 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-cb0d5029-123b-43de-ba1a-b871f4bbc1c1 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] Calling method '.version_select at 0x7f9da351c8b0>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:34.867584 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-cb0d5029-123b-43de-ba1a-b871f4bbc1c1 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] HTTP exception thrown: Volume 8b6148ad-20c2-449a-8810-adb993342031 could not be found. Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-cb0d5029-123b-43de-ba1a-b871f4bbc1c1 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] Request method failure captured: Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031 HTTP/1.1 Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYsvjy02UngPZppmoAdcWutEzj7ir9BNtCRFtGVoVzzMjLiv-DmlXkEVPVxQKnBxNEtHHoPhC60oU8jVpn5p8-JsPPnNXG9fyCYpuOQXkzwXHPPbP8JjynBGBTu316kLf5OmPrZA4O1ettI0bui4OLkH8ADYAH3zTrjSKFpSFg2cgpg Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 0ace1a5bdff14a648b45ebd4fa0429d4 Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-VolumesGetTestJSON-298614503 Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:32:34.870274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/0ace1a5bdff14a648b45ebd4fa0429d4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_0ace1a5bdff14a648b45ebd4fa0429d4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0ace1a5bdff14a648b45ebd4fa0429d4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/0ace1a5bdff14a648b45ebd4fa0429d4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-VolumesGetTestJSON-298614503 Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 0ace1a5bdff14a648b45ebd4fa0429d4 Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-VolumesGetTestJSON-298614503 Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-VolumesGetTestJSON-298614503-project Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 89eff7eb785648109cddaa69bde8430b Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-VolumesGetTestJSON-298614503-project Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Volume 8b6148ad-20c2-449a-8810-adb993342031 could not be found. Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '8b6148ad-20c2-449a-8810-adb993342031'} Aug 21 01:32:34.871783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 8b6148ad-20c2-449a-8810-adb993342031 could not be found. (HTTP 404) (Request-ID: req-ac92e6fb-7877-4b62-afea-b52f2fa6e060) Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:34.872719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 21 01:32:34.873972 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 8b6148ad-20c2-449a-8810-adb993342031 could not be found. Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 21 01:32:34.875536 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 8b6148ad-20c2-449a-8810-adb993342031 could not be found. Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 8b6148ad-20c2-449a-8810-adb993342031 could not be found. Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-cb0d5029-123b-43de-ba1a-b871f4bbc1c1 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] Returning 404 to user: Volume 8b6148ad-20c2-449a-8810-adb993342031 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-cb0d5029-123b-43de-ba1a-b871f4bbc1c1 tempest-VolumesGetTestJSON-298614503 tempest-VolumesGetTestJSON-298614503-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031" status: 404 len: 109 microversion: 2.1 time: 0.035494 Aug 21 01:32:34.876943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 458/901] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:32:34 2022] GET /compute/v2.1/os-volumes/8b6148ad-20c2-449a-8810-adb993342031 => generated 109 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 21 01:32:34.898883 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1012090006", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "99061e98-94a6-4cdf-b3c1-c9af69a6346c"}], "security_groups": [{"name": "tempest-securitygroup--1605226624"}], "key_name": "tempest-keypair-1210986866", "user_data": "}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:32:35.005163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] validate_networks() for [('99061e98-94a6-4cdf-b3c1-c9af69a6346c', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:32:35.193705 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=72,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:35.399572 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:32:35.400033 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:32:35.400536 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:32:35.401496 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:32:35.401939 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:32:35.411211 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:35.411728 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:35.412206 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:35.412967 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:35.413425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:35.413863 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:35.431338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for project 122b46d9e29843a88c7ca5f019426539. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:32:35.435301 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for user a91ac0ba7e8c4418930852b0fa184247 and project 122b46d9e29843a88c7ca5f019426539. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:32:35.446191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:32:35.448512 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:32:35.594061 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e6ea5a39-7a26-4fa4-8762-3374917be14a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.699844 Aug 21 01:32:35.594709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 444/902] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:32:34 2022] POST /compute/v2.1/servers => generated 402 bytes in 701 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:32:35.606897 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-965415cb-1c4f-41cc-a752-99683c44f778 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:35.607504 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-965415cb-1c4f-41cc-a752-99683c44f778 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:35.663005 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-965415cb-1c4f-41cc-a752-99683c44f778 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:35.670744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-965415cb-1c4f-41cc-a752-99683c44f778 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1242 microversion: 2.1 time: 0.068983 Aug 21 01:32:35.671185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 459/903] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:35 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1242 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:36.690942 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-53386c10-d366-4949-b0d0-4e56fff36eb5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:36.691654 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-53386c10-d366-4949-b0d0-4e56fff36eb5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:36.699753 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-53386c10-d366-4949-b0d0-4e56fff36eb5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:36.700380 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-53386c10-d366-4949-b0d0-4e56fff36eb5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:36.701150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-53386c10-d366-4949-b0d0-4e56fff36eb5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:36.705394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-53386c10-d366-4949-b0d0-4e56fff36eb5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:36.705904 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-53386c10-d366-4949-b0d0-4e56fff36eb5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:36.706298 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-53386c10-d366-4949-b0d0-4e56fff36eb5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:36.777821 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-53386c10-d366-4949-b0d0-4e56fff36eb5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:36.786543 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-53386c10-d366-4949-b0d0-4e56fff36eb5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1246 microversion: 2.1 time: 0.099579 Aug 21 01:32:36.787122 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 445/904] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:36 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1246 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:37.807695 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f3449eaa-53b0-4d4f-b817-5d424b5b43df tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:37.809135 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f3449eaa-53b0-4d4f-b817-5d424b5b43df tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:37.816047 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f3449eaa-53b0-4d4f-b817-5d424b5b43df tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:37.816573 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f3449eaa-53b0-4d4f-b817-5d424b5b43df tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:37.817031 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f3449eaa-53b0-4d4f-b817-5d424b5b43df tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:37.817729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f3449eaa-53b0-4d4f-b817-5d424b5b43df tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:37.818285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f3449eaa-53b0-4d4f-b817-5d424b5b43df tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:37.818672 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f3449eaa-53b0-4d4f-b817-5d424b5b43df tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:37.888677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f3449eaa-53b0-4d4f-b817-5d424b5b43df tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:37.897556 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f3449eaa-53b0-4d4f-b817-5d424b5b43df tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1294 microversion: 2.1 time: 0.094105 Aug 21 01:32:37.911651 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 460/905] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:37 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1294 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:38.919147 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-376d5434-bdcc-4024-8d40-dcf98b0c166d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:38.919871 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-376d5434-bdcc-4024-8d40-dcf98b0c166d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:38.927334 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-376d5434-bdcc-4024-8d40-dcf98b0c166d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:38.927564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-376d5434-bdcc-4024-8d40-dcf98b0c166d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:38.927856 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-376d5434-bdcc-4024-8d40-dcf98b0c166d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:38.928604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-376d5434-bdcc-4024-8d40-dcf98b0c166d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:38.929056 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-376d5434-bdcc-4024-8d40-dcf98b0c166d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:38.929483 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-376d5434-bdcc-4024-8d40-dcf98b0c166d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:39.112361 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-376d5434-bdcc-4024-8d40-dcf98b0c166d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:39.125166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-376d5434-bdcc-4024-8d40-dcf98b0c166d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1368 microversion: 2.1 time: 0.206322 Aug 21 01:32:39.125166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 446/906] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:38 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1368 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:39.828302 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-9164dfd1-5649-4102-a07b-4ae212ecb8c6 req-58409c4a-eb20-4d8e-975a-97581f2a073b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "049eb468-fdda-4d9d-a362-b13443842254", "tag": "0c27e3ca-1ce2-4451-9a8b-41f262f3e971"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:32:39.839177 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9164dfd1-5649-4102-a07b-4ae212ecb8c6 req-58409c4a-eb20-4d8e-975a-97581f2a073b service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:39.839741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9164dfd1-5649-4102-a07b-4ae212ecb8c6 req-58409c4a-eb20-4d8e-975a-97581f2a073b service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:39.840197 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9164dfd1-5649-4102-a07b-4ae212ecb8c6 req-58409c4a-eb20-4d8e-975a-97581f2a073b service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:39.851342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-9164dfd1-5649-4102-a07b-4ae212ecb8c6 req-58409c4a-eb20-4d8e-975a-97581f2a073b service nova] Creating event network-changed:0c27e3ca-1ce2-4451-9a8b-41f262f3e971 for instance 049eb468-fdda-4d9d-a362-b13443842254 on hv-cinder-maste Aug 21 01:32:39.856501 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-9164dfd1-5649-4102-a07b-4ae212ecb8c6 req-58409c4a-eb20-4d8e-975a-97581f2a073b service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031770 Aug 21 01:32:39.857115 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 461/907] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:32:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:32:40.145616 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-0cdb71b7-c9fb-4766-88fc-57cdcd7a2a0a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:40.146367 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-0cdb71b7-c9fb-4766-88fc-57cdcd7a2a0a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:40.153984 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0cdb71b7-c9fb-4766-88fc-57cdcd7a2a0a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:40.154571 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0cdb71b7-c9fb-4766-88fc-57cdcd7a2a0a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:40.155254 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0cdb71b7-c9fb-4766-88fc-57cdcd7a2a0a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:40.156043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0cdb71b7-c9fb-4766-88fc-57cdcd7a2a0a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:40.156671 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0cdb71b7-c9fb-4766-88fc-57cdcd7a2a0a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:40.157356 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0cdb71b7-c9fb-4766-88fc-57cdcd7a2a0a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:40.527090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0cdb71b7-c9fb-4766-88fc-57cdcd7a2a0a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:40.536974 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-0cdb71b7-c9fb-4766-88fc-57cdcd7a2a0a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1368 microversion: 2.1 time: 0.395895 Aug 21 01:32:40.537548 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 447/908] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:40 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1368 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:41.558839 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-94f8ca4a-ecdb-4c37-91f9-e097ce5e10b8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:41.559471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-94f8ca4a-ecdb-4c37-91f9-e097ce5e10b8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:41.565661 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-94f8ca4a-ecdb-4c37-91f9-e097ce5e10b8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:41.566145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-94f8ca4a-ecdb-4c37-91f9-e097ce5e10b8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:41.566578 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-94f8ca4a-ecdb-4c37-91f9-e097ce5e10b8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:41.567261 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-94f8ca4a-ecdb-4c37-91f9-e097ce5e10b8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:41.567712 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-94f8ca4a-ecdb-4c37-91f9-e097ce5e10b8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:41.568158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-94f8ca4a-ecdb-4c37-91f9-e097ce5e10b8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:41.703775 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-94f8ca4a-ecdb-4c37-91f9-e097ce5e10b8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:41.711736 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-94f8ca4a-ecdb-4c37-91f9-e097ce5e10b8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1368 microversion: 2.1 time: 0.158216 Aug 21 01:32:41.712359 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 462/909] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:41 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1368 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:42.732421 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-dc9738ec-1fc9-4537-845f-129da2fa7b83 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:42.733138 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-dc9738ec-1fc9-4537-845f-129da2fa7b83 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:42.739610 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dc9738ec-1fc9-4537-845f-129da2fa7b83 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:42.740266 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dc9738ec-1fc9-4537-845f-129da2fa7b83 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:42.740773 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dc9738ec-1fc9-4537-845f-129da2fa7b83 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:42.741561 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dc9738ec-1fc9-4537-845f-129da2fa7b83 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:42.742018 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dc9738ec-1fc9-4537-845f-129da2fa7b83 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:42.742437 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dc9738ec-1fc9-4537-845f-129da2fa7b83 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:42.877383 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-dc9738ec-1fc9-4537-845f-129da2fa7b83 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:42.885134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-dc9738ec-1fc9-4537-845f-129da2fa7b83 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1368 microversion: 2.1 time: 0.156754 Aug 21 01:32:42.888145 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 448/910] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:42 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1368 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:43.907001 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a75dae8c-8d11-4359-87c0-23fb4ffb60b7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:43.907616 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a75dae8c-8d11-4359-87c0-23fb4ffb60b7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:43.914405 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a75dae8c-8d11-4359-87c0-23fb4ffb60b7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:43.914891 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a75dae8c-8d11-4359-87c0-23fb4ffb60b7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:43.915359 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a75dae8c-8d11-4359-87c0-23fb4ffb60b7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:43.916048 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a75dae8c-8d11-4359-87c0-23fb4ffb60b7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:43.917149 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a75dae8c-8d11-4359-87c0-23fb4ffb60b7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:43.917700 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a75dae8c-8d11-4359-87c0-23fb4ffb60b7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:44.052294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a75dae8c-8d11-4359-87c0-23fb4ffb60b7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:44.060226 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a75dae8c-8d11-4359-87c0-23fb4ffb60b7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1368 microversion: 2.1 time: 0.157169 Aug 21 01:32:44.060822 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 463/911] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:43 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1368 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:45.081619 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-52dec92e-8adb-4c9f-a997-a08e16673188 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:45.082234 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-52dec92e-8adb-4c9f-a997-a08e16673188 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:45.088868 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-52dec92e-8adb-4c9f-a997-a08e16673188 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:45.092244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-52dec92e-8adb-4c9f-a997-a08e16673188 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:45.092244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-52dec92e-8adb-4c9f-a997-a08e16673188 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:45.092244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-52dec92e-8adb-4c9f-a997-a08e16673188 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:45.092244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-52dec92e-8adb-4c9f-a997-a08e16673188 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:45.092764 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-52dec92e-8adb-4c9f-a997-a08e16673188 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:45.230543 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-52dec92e-8adb-4c9f-a997-a08e16673188 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:45.238247 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-52dec92e-8adb-4c9f-a997-a08e16673188 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1368 microversion: 2.1 time: 0.160913 Aug 21 01:32:45.238814 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 449/912] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:45 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1368 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:45.762158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-c0b08fcb-4fb2-436b-ac59-ab8f039585db req-739ebc45-4f63-47c6-9d4c-086ed7337033 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "049eb468-fdda-4d9d-a362-b13443842254", "name": "network-vif-plugged", "status": "completed", "tag": "0c27e3ca-1ce2-4451-9a8b-41f262f3e971"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:32:45.773092 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:45.774383 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-c0b08fcb-4fb2-436b-ac59-ab8f039585db req-739ebc45-4f63-47c6-9d4c-086ed7337033 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:45.774870 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-c0b08fcb-4fb2-436b-ac59-ab8f039585db req-739ebc45-4f63-47c6-9d4c-086ed7337033 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:45.775309 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-c0b08fcb-4fb2-436b-ac59-ab8f039585db req-739ebc45-4f63-47c6-9d4c-086ed7337033 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:45.787432 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-c0b08fcb-4fb2-436b-ac59-ab8f039585db req-739ebc45-4f63-47c6-9d4c-086ed7337033 service nova] Creating event network-vif-plugged:0c27e3ca-1ce2-4451-9a8b-41f262f3e971 for instance 049eb468-fdda-4d9d-a362-b13443842254 on hv-cinder-maste Aug 21 01:32:45.796412 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-c0b08fcb-4fb2-436b-ac59-ab8f039585db req-739ebc45-4f63-47c6-9d4c-086ed7337033 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038705 Aug 21 01:32:45.797102 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 464/913] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:32:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:32:46.259809 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bc31cc08-6832-43cb-b1bc-abe8293d2ea8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:46.260561 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bc31cc08-6832-43cb-b1bc-abe8293d2ea8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:46.262632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:46.263485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=10 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:46.269514 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc31cc08-6832-43cb-b1bc-abe8293d2ea8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:46.270000 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc31cc08-6832-43cb-b1bc-abe8293d2ea8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:46.270439 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc31cc08-6832-43cb-b1bc-abe8293d2ea8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:46.271261 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc31cc08-6832-43cb-b1bc-abe8293d2ea8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:46.271710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc31cc08-6832-43cb-b1bc-abe8293d2ea8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:46.272151 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc31cc08-6832-43cb-b1bc-abe8293d2ea8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:46.406749 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-bc31cc08-6832-43cb-b1bc-abe8293d2ea8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:46.414852 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bc31cc08-6832-43cb-b1bc-abe8293d2ea8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1368 microversion: 2.1 time: 0.159513 Aug 21 01:32:46.423631 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 450/914] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:46 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1368 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:47.445009 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-71e4e68d-bd03-4061-8a2f-6e9228c8d218 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:47.445646 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-71e4e68d-bd03-4061-8a2f-6e9228c8d218 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:47.453040 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-71e4e68d-bd03-4061-8a2f-6e9228c8d218 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:47.453537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-71e4e68d-bd03-4061-8a2f-6e9228c8d218 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:47.453975 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-71e4e68d-bd03-4061-8a2f-6e9228c8d218 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:47.454667 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-71e4e68d-bd03-4061-8a2f-6e9228c8d218 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:47.455384 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-71e4e68d-bd03-4061-8a2f-6e9228c8d218 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:47.455814 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-71e4e68d-bd03-4061-8a2f-6e9228c8d218 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:47.585855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-71e4e68d-bd03-4061-8a2f-6e9228c8d218 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:47.593513 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-71e4e68d-bd03-4061-8a2f-6e9228c8d218 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1368 microversion: 2.1 time: 0.153610 Aug 21 01:32:47.594074 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 465/915] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:47 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1368 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:48.614163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-39251aee-f814-4c87-9a9f-a48f202de6ab tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:48.614710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-39251aee-f814-4c87-9a9f-a48f202de6ab tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:48.621793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-39251aee-f814-4c87-9a9f-a48f202de6ab tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:48.622275 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-39251aee-f814-4c87-9a9f-a48f202de6ab tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:48.622713 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-39251aee-f814-4c87-9a9f-a48f202de6ab tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:48.623405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-39251aee-f814-4c87-9a9f-a48f202de6ab tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:48.623861 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-39251aee-f814-4c87-9a9f-a48f202de6ab tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:48.624316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-39251aee-f814-4c87-9a9f-a48f202de6ab tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:48.763909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-39251aee-f814-4c87-9a9f-a48f202de6ab tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:48.771990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-39251aee-f814-4c87-9a9f-a48f202de6ab tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1557 microversion: 2.1 time: 0.161999 Aug 21 01:32:48.772622 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 451/916] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:32:48 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1557 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:48.794148 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-11b252b6-461a-4c76-bbbe-cb0510e6c2f9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:48.794711 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-11b252b6-461a-4c76-bbbe-cb0510e6c2f9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:48.801253 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-11b252b6-461a-4c76-bbbe-cb0510e6c2f9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:48.801739 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-11b252b6-461a-4c76-bbbe-cb0510e6c2f9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:48.802173 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-11b252b6-461a-4c76-bbbe-cb0510e6c2f9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:48.802870 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-11b252b6-461a-4c76-bbbe-cb0510e6c2f9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:48.803321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-11b252b6-461a-4c76-bbbe-cb0510e6c2f9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:48.803744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-11b252b6-461a-4c76-bbbe-cb0510e6c2f9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:48.892525 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-11b252b6-461a-4c76-bbbe-cb0510e6c2f9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.102215 Aug 21 01:32:48.893123 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 466/917] 10.222.0.47 () {58 vars in 1275 bytes} [Sun Aug 21 01:32:48 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254/os-interface => generated 284 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:32:49.960627 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-e8dc35a7-f7be-430f-8db6-6d796be369af req-f521b352-7a4a-4276-911b-f5468944352b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "049eb468-fdda-4d9d-a362-b13443842254", "tag": "0c27e3ca-1ce2-4451-9a8b-41f262f3e971"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:32:49.976765 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-e8dc35a7-f7be-430f-8db6-6d796be369af req-f521b352-7a4a-4276-911b-f5468944352b service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:49.976765 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-e8dc35a7-f7be-430f-8db6-6d796be369af req-f521b352-7a4a-4276-911b-f5468944352b service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:49.976765 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-e8dc35a7-f7be-430f-8db6-6d796be369af req-f521b352-7a4a-4276-911b-f5468944352b service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:49.987356 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-e8dc35a7-f7be-430f-8db6-6d796be369af req-f521b352-7a4a-4276-911b-f5468944352b service nova] Creating event network-changed:0c27e3ca-1ce2-4451-9a8b-41f262f3e971 for instance 049eb468-fdda-4d9d-a362-b13443842254 on hv-cinder-maste Aug 21 01:32:49.992597 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-e8dc35a7-f7be-430f-8db6-6d796be369af req-f521b352-7a4a-4276-911b-f5468944352b service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035676 Aug 21 01:32:49.993200 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 452/918] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:32:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:32:50.863525 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3b7b30a0-322e-4e39-a65f-3b1f9b39222f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:50.870925 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3b7b30a0-322e-4e39-a65f-3b1f9b39222f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:50.878790 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3b7b30a0-322e-4e39-a65f-3b1f9b39222f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:50.879279 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3b7b30a0-322e-4e39-a65f-3b1f9b39222f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:50.879719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3b7b30a0-322e-4e39-a65f-3b1f9b39222f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:50.880510 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3b7b30a0-322e-4e39-a65f-3b1f9b39222f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:50.881011 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3b7b30a0-322e-4e39-a65f-3b1f9b39222f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:50.881383 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3b7b30a0-322e-4e39-a65f-3b1f9b39222f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:50.885462 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=81,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:50.926008 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3b7b30a0-322e-4e39-a65f-3b1f9b39222f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.054893 Aug 21 01:32:50.926695 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 467/919] 10.222.0.47 () {60 vars in 1343 bytes} [Sun Aug 21 01:32:50 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a/os-volume_attachments => generated 25 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:32:50.994666 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7c58f0c1-d141-4321-bb46-7a6fd8c8c1cf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:50.995434 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7c58f0c1-d141-4321-bb46-7a6fd8c8c1cf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:51.002585 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7c58f0c1-d141-4321-bb46-7a6fd8c8c1cf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:51.003076 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7c58f0c1-d141-4321-bb46-7a6fd8c8c1cf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:51.003516 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7c58f0c1-d141-4321-bb46-7a6fd8c8c1cf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:51.004255 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7c58f0c1-d141-4321-bb46-7a6fd8c8c1cf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:51.004806 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7c58f0c1-d141-4321-bb46-7a6fd8c8c1cf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:51.005150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7c58f0c1-d141-4321-bb46-7a6fd8c8c1cf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:51.030853 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7c58f0c1-d141-4321-bb46-7a6fd8c8c1cf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:32:51.404049 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7c58f0c1-d141-4321-bb46-7a6fd8c8c1cf tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "DELETE /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 204 len: 0 microversion: 2.20 time: 0.413495 Aug 21 01:32:51.404620 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 453/920] 10.222.0.47 () {60 vars in 1280 bytes} [Sun Aug 21 01:32:50 2022] DELETE /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 0 bytes in 416 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 21 01:32:51.413297 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-434f5113-3681-4dad-9729-c5d47867c48b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:51.413917 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-434f5113-3681-4dad-9729-c5d47867c48b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:51.421406 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-434f5113-3681-4dad-9729-c5d47867c48b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:51.421887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-434f5113-3681-4dad-9729-c5d47867c48b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:51.422327 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-434f5113-3681-4dad-9729-c5d47867c48b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:51.423064 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-434f5113-3681-4dad-9729-c5d47867c48b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:51.423520 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-434f5113-3681-4dad-9729-c5d47867c48b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:51.423954 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-434f5113-3681-4dad-9729-c5d47867c48b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:51.565114 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-434f5113-3681-4dad-9729-c5d47867c48b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:51.573797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-434f5113-3681-4dad-9729-c5d47867c48b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:51.575887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-434f5113-3681-4dad-9729-c5d47867c48b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:51.577111 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-434f5113-3681-4dad-9729-c5d47867c48b tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1720 microversion: 2.20 time: 0.168175 Aug 21 01:32:51.577682 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 468/921] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:51 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1720 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:52.601538 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-dfad23ef-f035-4178-bb06-42daee561ce0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:52.602606 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-dfad23ef-f035-4178-bb06-42daee561ce0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:52.609975 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dfad23ef-f035-4178-bb06-42daee561ce0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:52.610469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dfad23ef-f035-4178-bb06-42daee561ce0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:52.610916 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dfad23ef-f035-4178-bb06-42daee561ce0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:52.611620 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dfad23ef-f035-4178-bb06-42daee561ce0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:52.612123 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dfad23ef-f035-4178-bb06-42daee561ce0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:52.612562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-dfad23ef-f035-4178-bb06-42daee561ce0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:52.616606 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:52.751512 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-dfad23ef-f035-4178-bb06-42daee561ce0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:52.760612 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-dfad23ef-f035-4178-bb06-42daee561ce0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:52.762156 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-dfad23ef-f035-4178-bb06-42daee561ce0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:52.763344 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-dfad23ef-f035-4178-bb06-42daee561ce0 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1720 microversion: 2.20 time: 0.166081 Aug 21 01:32:52.763893 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 454/922] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:52 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1720 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 21 01:32:53.799873 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f8a55499-793f-477e-bc6d-83ea7d6053c1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:53.800941 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f8a55499-793f-477e-bc6d-83ea7d6053c1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:53.810097 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8a55499-793f-477e-bc6d-83ea7d6053c1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:53.810955 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8a55499-793f-477e-bc6d-83ea7d6053c1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:53.811652 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8a55499-793f-477e-bc6d-83ea7d6053c1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:53.812745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8a55499-793f-477e-bc6d-83ea7d6053c1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:53.813389 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8a55499-793f-477e-bc6d-83ea7d6053c1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:53.814036 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f8a55499-793f-477e-bc6d-83ea7d6053c1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:53.939686 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f8a55499-793f-477e-bc6d-83ea7d6053c1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:53.949031 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f8a55499-793f-477e-bc6d-83ea7d6053c1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:53.950639 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f8a55499-793f-477e-bc6d-83ea7d6053c1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f4a35c46e4741c1b238f24d203b416d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13b534c485db48d8a37e024702f87092', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:32:53.951814 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f8a55499-793f-477e-bc6d-83ea7d6053c1 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 200 len: 1653 microversion: 2.20 time: 0.158023 Aug 21 01:32:53.952415 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 469/923] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:53 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 1653 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 21 01:32:54.166074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-032c1ebf-6bf3-4ec9-820a-1a83d8466f9a req-8556dec9-982a-4425-8dd6-e54d380ef287 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a5b475aa-486a-4957-8b54-aa1ee7438a8a", "name": "network-vif-deleted", "tag": "d3beeb80-5980-4de1-b832-2d4a1e1dae1c"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:32:54.178381 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-032c1ebf-6bf3-4ec9-820a-1a83d8466f9a req-8556dec9-982a-4425-8dd6-e54d380ef287 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:54.179098 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-032c1ebf-6bf3-4ec9-820a-1a83d8466f9a req-8556dec9-982a-4425-8dd6-e54d380ef287 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:54.179505 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-032c1ebf-6bf3-4ec9-820a-1a83d8466f9a req-8556dec9-982a-4425-8dd6-e54d380ef287 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:54.192299 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-032c1ebf-6bf3-4ec9-820a-1a83d8466f9a req-8556dec9-982a-4425-8dd6-e54d380ef287 service nova] Creating event network-vif-deleted:d3beeb80-5980-4de1-b832-2d4a1e1dae1c for instance a5b475aa-486a-4957-8b54-aa1ee7438a8a on hv-cinder-maste Aug 21 01:32:54.197961 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-032c1ebf-6bf3-4ec9-820a-1a83d8466f9a req-8556dec9-982a-4425-8dd6-e54d380ef287 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035609 Aug 21 01:32:54.199209 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 455/924] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:32:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:32:54.975140 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7e7e46b0-607f-4182-bf32-ff87b075757f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:54.976279 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7e7e46b0-607f-4182-bf32-ff87b075757f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:54.987494 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7e7e46b0-607f-4182-bf32-ff87b075757f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:54.988824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7e7e46b0-607f-4182-bf32-ff87b075757f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:54.990271 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7e7e46b0-607f-4182-bf32-ff87b075757f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:54.991383 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7e7e46b0-607f-4182-bf32-ff87b075757f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:54.991782 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7e7e46b0-607f-4182-bf32-ff87b075757f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:54.992288 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7e7e46b0-607f-4182-bf32-ff87b075757f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:55.011011 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-7e7e46b0-607f-4182-bf32-ff87b075757f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] HTTP exception thrown: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7e7e46b0-607f-4182-bf32-ff87b075757f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Request method failure captured: Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a HTTP/1.1 Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYl-urYu2WLpkg5KrWqi8t_1VSj0UQHumB0Ida0LouNet0l3Iiuz9dGUg5N8KGW-c2ywdymoBqXU-bzK3wT0eSO35h49EIb-38apkB4ujX4cihQ9PRiPI78eT9VuEcB3HW4C8TNVfy-LXwDEz2Bxg2owaQkc9JnefnISkKvong0WKOk Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Openstack-Nova-Api-Version: 2.20 Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 13b534c485db48d8a37e024702f87092 Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:32:55.013434 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:32:55.015299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/13b534c485db48d8a37e024702f87092"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_13b534c485db48d8a37e024702f87092"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/13b534c485db48d8a37e024702f87092"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/13b534c485db48d8a37e024702f87092"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:32:55.015299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:55.015299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 13b534c485db48d8a37e024702f87092 Aug 21 01:32:55.015299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:55.015299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeShelveTestJSON-1747916427-project Aug 21 01:32:55.015299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:32:55.015299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:32:55.015299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 0f4a35c46e4741c1b238f24d203b416d Aug 21 01:32:55.015299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1747916427-project Aug 21 01:32:55.015299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:32:55.015299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:55.015299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': 'a5b475aa-486a-4957-8b54-aa1ee7438a8a'} Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:32:55.019855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:55.021321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:55.022471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7e7e46b0-607f-4182-bf32-ff87b075757f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Returning 404 to user: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:32:55.022471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7e7e46b0-607f-4182-bf32-ff87b075757f tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 404 len: 111 microversion: 2.20 time: 0.043871 Aug 21 01:32:55.022471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 470/925] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:54 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 21 01:32:55.026841 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-33f4f3ef-4572-4d7d-80e7-7c1b5ff02eac tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '.version_select at 0x7f9da32e19d0>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:55.155829 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-33f4f3ef-4572-4d7d-80e7-7c1b5ff02eac tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-338112113" status: 204 len: 0 microversion: 2.20 time: 0.133567 Aug 21 01:32:55.156864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 456/926] 10.222.0.47 () {60 vars in 1259 bytes} [Sun Aug 21 01:32:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-338112113 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 21 01:32:57.621948 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-69893973-c041-4ac2-af32-42016ff809f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:57.622518 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-69893973-c041-4ac2-af32-42016ff809f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:57.625218 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=38 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:32:57.630084 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-69893973-c041-4ac2-af32-42016ff809f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:57.630577 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-69893973-c041-4ac2-af32-42016ff809f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:57.631013 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-69893973-c041-4ac2-af32-42016ff809f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:57.631723 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-69893973-c041-4ac2-af32-42016ff809f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:57.632387 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-69893973-c041-4ac2-af32-42016ff809f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:57.632823 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-69893973-c041-4ac2-af32-42016ff809f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:57.647670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-69893973-c041-4ac2-af32-42016ff809f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] HTTP exception thrown: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-69893973-c041-4ac2-af32-42016ff809f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Request method failure captured: Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: DELETE /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a HTTP/1.1 Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYl-urYu2WLpkg5KrWqi8t_1VSj0UQHumB0Ida0LouNet0l3Iiuz9dGUg5N8KGW-c2ywdymoBqXU-bzK3wT0eSO35h49EIb-38apkB4ujX4cihQ9PRiPI78eT9VuEcB3HW4C8TNVfy-LXwDEz2Bxg2owaQkc9JnefnISkKvong0WKOk Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Openstack-Nova-Api-Version: 2.20 Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 13b534c485db48d8a37e024702f87092 Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:32:57.649398 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:32:57.651225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/13b534c485db48d8a37e024702f87092"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_13b534c485db48d8a37e024702f87092"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/13b534c485db48d8a37e024702f87092"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/13b534c485db48d8a37e024702f87092"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:32:57.651225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:57.651225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 13b534c485db48d8a37e024702f87092 Aug 21 01:32:57.651225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:57.651225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeShelveTestJSON-1747916427-project Aug 21 01:32:57.651225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:32:57.651225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:32:57.651225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 0f4a35c46e4741c1b238f24d203b416d Aug 21 01:32:57.651225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1747916427-project Aug 21 01:32:57.651225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:32:57.651225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:57.651225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': 'a5b475aa-486a-4957-8b54-aa1ee7438a8a'} Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:32:57.652433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:57.654543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:57.656193 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-69893973-c041-4ac2-af32-42016ff809f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Returning 404 to user: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:32:57.656193 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-69893973-c041-4ac2-af32-42016ff809f3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "DELETE /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 404 len: 111 microversion: 2.20 time: 0.033836 Aug 21 01:32:57.656193 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 471/927] 10.222.0.47 () {60 vars in 1280 bytes} [Sun Aug 21 01:32:57 2022] DELETE /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 21 01:32:57.660253 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-870cafa2-cee3-45e2-94ac-d8fb35cf11b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:57.660995 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-870cafa2-cee3-45e2-94ac-d8fb35cf11b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: a5b475aa-486a-4957-8b54-aa1ee7438a8a] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:57.667945 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-870cafa2-cee3-45e2-94ac-d8fb35cf11b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:57.668513 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-870cafa2-cee3-45e2-94ac-d8fb35cf11b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:57.668990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-870cafa2-cee3-45e2-94ac-d8fb35cf11b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:57.670009 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-870cafa2-cee3-45e2-94ac-d8fb35cf11b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:57.670243 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-870cafa2-cee3-45e2-94ac-d8fb35cf11b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:57.670674 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-870cafa2-cee3-45e2-94ac-d8fb35cf11b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:57.686375 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-870cafa2-cee3-45e2-94ac-d8fb35cf11b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] HTTP exception thrown: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-870cafa2-cee3-45e2-94ac-d8fb35cf11b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Request method failure captured: Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a HTTP/1.1 Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYl-urYu2WLpkg5KrWqi8t_1VSj0UQHumB0Ida0LouNet0l3Iiuz9dGUg5N8KGW-c2ywdymoBqXU-bzK3wT0eSO35h49EIb-38apkB4ujX4cihQ9PRiPI78eT9VuEcB3HW4C8TNVfy-LXwDEz2Bxg2owaQkc9JnefnISkKvong0WKOk Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Openstack-Nova-Api-Version: 2.20 Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 13b534c485db48d8a37e024702f87092 Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:32:57.688346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:32:57.690045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/13b534c485db48d8a37e024702f87092"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_13b534c485db48d8a37e024702f87092"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/13b534c485db48d8a37e024702f87092"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/13b534c485db48d8a37e024702f87092"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:32:57.690045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:57.690045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 13b534c485db48d8a37e024702f87092 Aug 21 01:32:57.690045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:57.690045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeShelveTestJSON-1747916427-project Aug 21 01:32:57.690045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:32:57.690045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:32:57.690045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: 0f4a35c46e4741c1b238f24d203b416d Aug 21 01:32:57.690045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1747916427-project Aug 21 01:32:57.690045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:32:57.690045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:57.690045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': 'a5b475aa-486a-4957-8b54-aa1ee7438a8a'} Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:32:57.691129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. Aug 21 01:32:57.692733 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:57.694121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-870cafa2-cee3-45e2-94ac-d8fb35cf11b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Returning 404 to user: Instance a5b475aa-486a-4957-8b54-aa1ee7438a8a could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:32:57.694121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-870cafa2-cee3-45e2-94ac-d8fb35cf11b3 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a" status: 404 len: 111 microversion: 2.20 time: 0.033416 Aug 21 01:32:57.694121 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 457/928] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:57 2022] GET /compute/v2.1/servers/a5b475aa-486a-4957-8b54-aa1ee7438a8a => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 21 01:32:58.935617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e145dcd3-5003-4cbb-a2f6-4c38c0333cc2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:58.936907 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e145dcd3-5003-4cbb-a2f6-4c38c0333cc2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:58.944491 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e145dcd3-5003-4cbb-a2f6-4c38c0333cc2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:58.948843 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e145dcd3-5003-4cbb-a2f6-4c38c0333cc2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:58.948843 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e145dcd3-5003-4cbb-a2f6-4c38c0333cc2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:58.948843 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e145dcd3-5003-4cbb-a2f6-4c38c0333cc2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:58.948843 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e145dcd3-5003-4cbb-a2f6-4c38c0333cc2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:58.949265 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e145dcd3-5003-4cbb-a2f6-4c38c0333cc2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:58.961603 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-e145dcd3-5003-4cbb-a2f6-4c38c0333cc2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] HTTP exception thrown: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e145dcd3-5003-4cbb-a2f6-4c38c0333cc2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Request method failure captured: Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: DELETE /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 HTTP/1.1 Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYl-urYu2WLpkg5KrWqi8t_1VSj0UQHumB0Ida0LouNet0l3Iiuz9dGUg5N8KGW-c2ywdymoBqXU-bzK3wT0eSO35h49EIb-38apkB4ujX4cihQ9PRiPI78eT9VuEcB3HW4C8TNVfy-LXwDEz2Bxg2owaQkc9JnefnISkKvong0WKOk Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Openstack-Nova-Api-Version: 2.20 Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 13b534c485db48d8a37e024702f87092 Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:32:58.963183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:32:58.964906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/13b534c485db48d8a37e024702f87092"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_13b534c485db48d8a37e024702f87092"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/13b534c485db48d8a37e024702f87092"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/13b534c485db48d8a37e024702f87092"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:32:58.964906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:58.964906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 13b534c485db48d8a37e024702f87092 Aug 21 01:32:58.964906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:58.964906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeShelveTestJSON-1747916427-project Aug 21 01:32:58.964906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:32:58.964906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:32:58.964906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 0f4a35c46e4741c1b238f24d203b416d Aug 21 01:32:58.964906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1747916427-project Aug 21 01:32:58.964906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:32:58.964906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:32:58.964906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1'} Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:32:58.965993 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:32:58.967334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:32:58.968593 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e145dcd3-5003-4cbb-a2f6-4c38c0333cc2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Returning 404 to user: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:32:58.968593 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e145dcd3-5003-4cbb-a2f6-4c38c0333cc2 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "DELETE /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 404 len: 111 microversion: 2.20 time: 0.032962 Aug 21 01:32:58.968593 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 472/929] 10.222.0.47 () {60 vars in 1280 bytes} [Sun Aug 21 01:32:58 2022] DELETE /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 21 01:32:58.974002 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f59ad9f2-945a-4632-b2ae-05f2ab40fe03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:32:58.977570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f59ad9f2-945a-4632-b2ae-05f2ab40fe03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] [instance: 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:32:58.984570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f59ad9f2-945a-4632-b2ae-05f2ab40fe03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:58.986162 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f59ad9f2-945a-4632-b2ae-05f2ab40fe03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:58.986162 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f59ad9f2-945a-4632-b2ae-05f2ab40fe03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:58.987009 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f59ad9f2-945a-4632-b2ae-05f2ab40fe03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:32:58.987009 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f59ad9f2-945a-4632-b2ae-05f2ab40fe03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:32:58.987205 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f59ad9f2-945a-4632-b2ae-05f2ab40fe03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:32:59.002482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-f59ad9f2-945a-4632-b2ae-05f2ab40fe03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] HTTP exception thrown: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f59ad9f2-945a-4632-b2ae-05f2ab40fe03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Request method failure captured: Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 HTTP/1.1 Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYl-urYu2WLpkg5KrWqi8t_1VSj0UQHumB0Ida0LouNet0l3Iiuz9dGUg5N8KGW-c2ywdymoBqXU-bzK3wT0eSO35h49EIb-38apkB4ujX4cihQ9PRiPI78eT9VuEcB3HW4C8TNVfy-LXwDEz2Bxg2owaQkc9JnefnISkKvong0WKOk Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Openstack-Nova-Api-Version: 2.20 Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 13b534c485db48d8a37e024702f87092 Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:32:59.004023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:32:59.007575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/13b534c485db48d8a37e024702f87092"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_13b534c485db48d8a37e024702f87092"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/13b534c485db48d8a37e024702f87092"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/13b534c485db48d8a37e024702f87092"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:32:59.007575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:59.007575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 13b534c485db48d8a37e024702f87092 Aug 21 01:32:59.007575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1747916427 Aug 21 01:32:59.007575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeShelveTestJSON-1747916427-project Aug 21 01:32:59.007575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:32:59.007575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:32:59.007575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: 0f4a35c46e4741c1b238f24d203b416d Aug 21 01:32:59.007575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1747916427-project Aug 21 01:32:59.007575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:32:59.007575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:32:59.007575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1'} Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:32:59.008520 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. Aug 21 01:32:59.010006 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:32:59.011766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f59ad9f2-945a-4632-b2ae-05f2ab40fe03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] Returning 404 to user: Instance 2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:32:59.011766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f59ad9f2-945a-4632-b2ae-05f2ab40fe03 tempest-AttachVolumeShelveTestJSON-1747916427 tempest-AttachVolumeShelveTestJSON-1747916427-project] 10.222.0.47 "GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1" status: 404 len: 111 microversion: 2.20 time: 0.035468 Aug 21 01:32:59.011766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 458/930] 10.222.0.47 () {60 vars in 1277 bytes} [Sun Aug 21 01:32:58 2022] GET /compute/v2.1/servers/2fb6d87e-f5bc-4612-bf7a-4204e1e45bb1 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 21 01:33:06.602879 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-36c545e7-b02b-4e8d-be2c-90964165d1dd tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] Calling method '.version_select at 0x7f9da32635e0>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:06.651525 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-36c545e7-b02b-4e8d-be2c-90964165d1dd tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/8a0c064e-1bcf-471f-8af3-3ce3642196da" status: 200 len: 350 microversion: 2.1 time: 0.052327 Aug 21 01:33:06.652171 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 473/931] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:33:06 2022] GET /compute/v2.1/os-volumes/8a0c064e-1bcf-471f-8af3-3ce3642196da => generated 350 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:33:08.463315 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fb8bd35c-97f7-48b5-9484-f7bb2d3edb75 tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] Calling method '.version_select at 0x7f9da3385c10>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:08.513051 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fb8bd35c-97f7-48b5-9484-f7bb2d3edb75 tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/dcf5262e-b350-4c3c-8b30-77b3f3866b4e" status: 200 len: 350 microversion: 2.1 time: 0.053613 Aug 21 01:33:08.513899 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 459/932] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:33:08 2022] GET /compute/v2.1/os-volumes/dcf5262e-b350-4c3c-8b30-77b3f3866b4e => generated 350 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:10.125806 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=3 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:33:10.127134 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=23 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:33:10.130980 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a31617a9-a23f-446e-9312-c374db3e3746 tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] Calling method '.version_select at 0x7f9da34ea430>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:10.184987 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a31617a9-a23f-446e-9312-c374db3e3746 tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/94f7f9fa-2126-4499-a759-6191e717595f" status: 200 len: 350 microversion: 2.1 time: 0.060310 Aug 21 01:33:10.185626 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 474/933] 10.222.0.47 () {58 vars in 1245 bytes} [Sun Aug 21 01:33:10 2022] GET /compute/v2.1/os-volumes/94f7f9fa-2126-4499-a759-6191e717595f => generated 350 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:33:10.203655 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2094c030-f7eb-4a06-a097-b7935808022c tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] Calling method '.version_select at 0x7f9da32e1430>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:10.206724 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=46,nova_api:INSERT=3,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:33:10.207570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=11 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:33:10.284727 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2094c030-f7eb-4a06-a097-b7935808022c tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] 10.222.0.47 "GET /compute/v2.1/os-volumes" status: 200 len: 1033 microversion: 2.1 time: 0.090182 Aug 21 01:33:10.285287 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 460/934] 10.222.0.47 () {58 vars in 1134 bytes} [Sun Aug 21 01:33:10 2022] GET /compute/v2.1/os-volumes => generated 1033 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:10.299696 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f587c8f4-d1a2-4dcb-b3ec-5b60487ee8b5 tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] Calling method '.version_select at 0x7f9da3263430>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:10.384850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f587c8f4-d1a2-4dcb-b3ec-5b60487ee8b5 tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] 10.222.0.47 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.089950 Aug 21 01:33:10.385425 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 475/935] 10.222.0.47 () {58 vars in 1149 bytes} [Sun Aug 21 01:33:10 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 693 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:10.399617 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-40d28d82-aa25-4810-be4a-2019658f8260 tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] Calling method '.version_select at 0x7f9da3385c10>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:10.483470 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-40d28d82-aa25-4810-be4a-2019658f8260 tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] 10.222.0.47 "GET /compute/v2.1/os-volumes" status: 200 len: 1033 microversion: 2.1 time: 0.088103 Aug 21 01:33:10.484715 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 461/936] 10.222.0.47 () {58 vars in 1134 bytes} [Sun Aug 21 01:33:10 2022] GET /compute/v2.1/os-volumes => generated 1033 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:10.497863 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-04947ec1-8a6c-4d61-a6ad-9d7ecc70731e tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] Calling method '.version_select at 0x7f9da32de670>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:10.583632 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-04947ec1-8a6c-4d61-a6ad-9d7ecc70731e tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] 10.222.0.47 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.089521 Aug 21 01:33:10.584246 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 476/937] 10.222.0.47 () {58 vars in 1167 bytes} [Sun Aug 21 01:33:10 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:10.598092 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-674f015e-b979-49fe-a898-bdd5a5ced6cd tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] Calling method '.version_select at 0x7f9da32f6af0>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:10.680727 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-674f015e-b979-49fe-a898-bdd5a5ced6cd tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.087207 Aug 21 01:33:10.681299 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 462/938] 10.222.0.47 () {58 vars in 1170 bytes} [Sun Aug 21 01:33:10 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 693 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:10.695307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c506e93c-5548-4c00-96cc-97744d89def1 tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] Calling method '.version_select at 0x7f9da31a9a60>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:10.777532 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c506e93c-5548-4c00-96cc-97744d89def1 tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1033 microversion: 2.1 time: 0.086264 Aug 21 01:33:10.778236 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 477/939] 10.222.0.47 () {58 vars in 1155 bytes} [Sun Aug 21 01:33:10 2022] GET /compute/v2.1/os-volumes/detail => generated 1033 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:10.791799 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3b62cf68-8406-4ae8-95a7-d7471b1370bb tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] Calling method '.version_select at 0x7f9da32a0ca0>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:10.877407 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3b62cf68-8406-4ae8-95a7-d7471b1370bb tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.089504 Aug 21 01:33:10.877981 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 463/940] 10.222.0.47 () {58 vars in 1188 bytes} [Sun Aug 21 01:33:10 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:10.892630 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d1aaad11-5b2f-4cb5-9c51-277db5a0257d tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] Calling method '.version_select at 0x7f9da31a9430>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:10.974587 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d1aaad11-5b2f-4cb5-9c51-277db5a0257d tempest-VolumesTestJSON-720924148 tempest-VolumesTestJSON-720924148-project] 10.222.0.47 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1033 microversion: 2.1 time: 0.086362 Aug 21 01:33:10.975186 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 478/941] 10.222.0.47 () {58 vars in 1155 bytes} [Sun Aug 21 01:33:10 2022] GET /compute/v2.1/os-volumes/detail => generated 1033 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:33.435122 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-212f35cd-2374-47e0-adaa-92a4d2cd2524 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:33.435661 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-212f35cd-2374-47e0-adaa-92a4d2cd2524 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:33.441723 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-212f35cd-2374-47e0-adaa-92a4d2cd2524 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:33.442383 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-212f35cd-2374-47e0-adaa-92a4d2cd2524 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:33.442655 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-212f35cd-2374-47e0-adaa-92a4d2cd2524 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:33.443852 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-212f35cd-2374-47e0-adaa-92a4d2cd2524 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:33.444346 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-212f35cd-2374-47e0-adaa-92a4d2cd2524 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:33.444780 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-212f35cd-2374-47e0-adaa-92a4d2cd2524 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:33.468902 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-212f35cd-2374-47e0-adaa-92a4d2cd2524 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254/ips" status: 200 len: 149 microversion: 2.1 time: 0.038429 Aug 21 01:33:33.469485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 464/942] 10.222.0.47 () {58 vars in 1248 bytes} [Sun Aug 21 01:33:33 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254/ips => generated 149 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:33.482649 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7eb7dafb-17cf-4362-a223-5d7ea5c46f52 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "fc6ceacd-e32a-4de3-bd34-144c0e60919d"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:33:33.485610 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7eb7dafb-17cf-4362-a223-5d7ea5c46f52 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:33.492958 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7eb7dafb-17cf-4362-a223-5d7ea5c46f52 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:33.493440 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7eb7dafb-17cf-4362-a223-5d7ea5c46f52 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:33.493870 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7eb7dafb-17cf-4362-a223-5d7ea5c46f52 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:33.494564 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7eb7dafb-17cf-4362-a223-5d7ea5c46f52 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:33.495008 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7eb7dafb-17cf-4362-a223-5d7ea5c46f52 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:33.495592 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7eb7dafb-17cf-4362-a223-5d7ea5c46f52 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:33.576728 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-7eb7dafb-17cf-4362-a223-5d7ea5c46f52 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Invalid volume: volume fc6ceacd-e32a-4de3-bd34-144c0e60919d is already attached to instances: d1a9e859-eddf-4f45-8e02-168201b29a63 Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7eb7dafb-17cf-4362-a223-5d7ea5c46f52 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: POST /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254/os-volume_attachments HTTP/1.1 Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 74 Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:33:33.578459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:33:33.580076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:33:33.580076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:33:33.580076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:33:33.580076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:33:33.580076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:33:33.580076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:33:33.580076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:33:33.580076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:33:33.580076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:33:33.580076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Aug 21 01:33:33.580076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {"volumeAttachment": {"volumeId": "fc6ceacd-e32a-4de3-bd34-144c0e60919d"}} Aug 21 01:33:33.580076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Invalid volume: volume fc6ceacd-e32a-4de3-bd34-144c0e60919d is already attached to instances: d1a9e859-eddf-4f45-8e02-168201b29a63 Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'server_id': '049eb468-fdda-4d9d-a362-b13443842254', 'body': {'volumeAttachment': {'volumeId': 'fc6ceacd-e32a-4de3-bd34-144c0e60919d'}}} Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5113, in attach_volume Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4953, in _check_volume_already_attached Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume fc6ceacd-e32a-4de3-bd34-144c0e60919d is already attached to instances: d1a9e859-eddf-4f45-8e02-168201b29a63 Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:33:33.581371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume fc6ceacd-e32a-4de3-bd34-144c0e60919d is already attached to instances: d1a9e859-eddf-4f45-8e02-168201b29a63 Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:33:33.582950 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:33:33.585726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:33:33.585726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:33:33.585726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:33:33.585726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume fc6ceacd-e32a-4de3-bd34-144c0e60919d is already attached to instances: d1a9e859-eddf-4f45-8e02-168201b29a63 Aug 21 01:33:33.585726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:33:33.585726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7eb7dafb-17cf-4362-a223-5d7ea5c46f52 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 400 to user: Invalid volume: volume fc6ceacd-e32a-4de3-bd34-144c0e60919d is already attached to instances: d1a9e859-eddf-4f45-8e02-168201b29a63 {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:33:33.585726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7eb7dafb-17cf-4362-a223-5d7ea5c46f52 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.102605 Aug 21 01:33:33.585726 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 479/943] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:33:33 2022] POST /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254/os-volume_attachments => generated 174 bytes in 104 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 21 01:33:33.591580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-55892c72-c624-4279-bd0c-0d05494e6ef5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:33.592311 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-55892c72-c624-4279-bd0c-0d05494e6ef5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:33.598508 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55892c72-c624-4279-bd0c-0d05494e6ef5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:33.599008 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55892c72-c624-4279-bd0c-0d05494e6ef5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:33.599501 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55892c72-c624-4279-bd0c-0d05494e6ef5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:33.600264 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55892c72-c624-4279-bd0c-0d05494e6ef5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:33.600734 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55892c72-c624-4279-bd0c-0d05494e6ef5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:33.601162 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55892c72-c624-4279-bd0c-0d05494e6ef5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:33.631355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-55892c72-c624-4279-bd0c-0d05494e6ef5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:33:33.907884 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-55892c72-c624-4279-bd0c-0d05494e6ef5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 204 len: 0 microversion: 2.1 time: 0.319191 Aug 21 01:33:33.908011 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 465/944] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:33:33 2022] DELETE /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:33:33.916553 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c278257a-d473-4da1-97d7-3673a01e2863 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:33.920040 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c278257a-d473-4da1-97d7-3673a01e2863 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:33.924631 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c278257a-d473-4da1-97d7-3673a01e2863 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:33.925227 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c278257a-d473-4da1-97d7-3673a01e2863 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:33.925791 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c278257a-d473-4da1-97d7-3673a01e2863 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:33.926503 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c278257a-d473-4da1-97d7-3673a01e2863 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:33.926962 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c278257a-d473-4da1-97d7-3673a01e2863 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:33.927949 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c278257a-d473-4da1-97d7-3673a01e2863 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:34.067850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c278257a-d473-4da1-97d7-3673a01e2863 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:34.079946 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c278257a-d473-4da1-97d7-3673a01e2863 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1682 microversion: 2.1 time: 0.167437 Aug 21 01:33:34.080886 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 480/945] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:33 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1682 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:35.104366 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f6b37452-dcdd-48a8-8b73-ac4a3a7d3385 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:35.104366 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f6b37452-dcdd-48a8-8b73-ac4a3a7d3385 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:35.110377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f6b37452-dcdd-48a8-8b73-ac4a3a7d3385 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:35.110925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f6b37452-dcdd-48a8-8b73-ac4a3a7d3385 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:35.111436 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f6b37452-dcdd-48a8-8b73-ac4a3a7d3385 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:35.112221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f6b37452-dcdd-48a8-8b73-ac4a3a7d3385 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:35.112739 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f6b37452-dcdd-48a8-8b73-ac4a3a7d3385 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:35.113228 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f6b37452-dcdd-48a8-8b73-ac4a3a7d3385 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:35.251420 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f6b37452-dcdd-48a8-8b73-ac4a3a7d3385 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:35.260044 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f6b37452-dcdd-48a8-8b73-ac4a3a7d3385 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1682 microversion: 2.1 time: 0.160913 Aug 21 01:33:35.260627 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 466/946] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:35 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1682 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:35.955453 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:33:35.955453 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Sun Aug 21 01:33:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:33:36.189170 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:33:36.189467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Sun Aug 21 01:33:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:33:36.281182 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-89c297ee-95f0-42ce-803d-ce9e719e298f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:36.281945 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-89c297ee-95f0-42ce-803d-ce9e719e298f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:36.288826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-89c297ee-95f0-42ce-803d-ce9e719e298f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:36.289329 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-89c297ee-95f0-42ce-803d-ce9e719e298f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:36.289961 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-89c297ee-95f0-42ce-803d-ce9e719e298f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:36.290688 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-89c297ee-95f0-42ce-803d-ce9e719e298f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:36.291147 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-89c297ee-95f0-42ce-803d-ce9e719e298f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:36.291753 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-89c297ee-95f0-42ce-803d-ce9e719e298f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:36.420573 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-89c297ee-95f0-42ce-803d-ce9e719e298f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:36.429094 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-89c297ee-95f0-42ce-803d-ce9e719e298f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1614 microversion: 2.1 time: 0.151616 Aug 21 01:33:36.429980 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 481/947] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:36 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1614 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:36.638270 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-9c8c5806-384d-4d84-854f-02f98a80466c req-51855aaf-d8ed-4f06-90b3-ea24d3ffbb9a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "049eb468-fdda-4d9d-a362-b13443842254", "name": "network-vif-deleted", "tag": "0c27e3ca-1ce2-4451-9a8b-41f262f3e971"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:33:36.656462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-9c8c5806-384d-4d84-854f-02f98a80466c req-51855aaf-d8ed-4f06-90b3-ea24d3ffbb9a service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:36.657745 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-9c8c5806-384d-4d84-854f-02f98a80466c req-51855aaf-d8ed-4f06-90b3-ea24d3ffbb9a service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:36.658342 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-9c8c5806-384d-4d84-854f-02f98a80466c req-51855aaf-d8ed-4f06-90b3-ea24d3ffbb9a service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:36.695470 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-9c8c5806-384d-4d84-854f-02f98a80466c req-51855aaf-d8ed-4f06-90b3-ea24d3ffbb9a service nova] Creating event network-vif-deleted:0c27e3ca-1ce2-4451-9a8b-41f262f3e971 for instance 049eb468-fdda-4d9d-a362-b13443842254 on hv-cinder-maste Aug 21 01:33:36.703840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-9c8c5806-384d-4d84-854f-02f98a80466c req-51855aaf-d8ed-4f06-90b3-ea24d3ffbb9a service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069475 Aug 21 01:33:36.719198 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 467/948] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:33:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:37.450329 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:37.450960 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:37.457519 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:37.458014 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:37.459204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:37.467382 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:37.467382 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:37.467382 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:37.497727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:37.498328 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:37.498727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:37.499185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lazy-loading 'fault' on Instance uuid 049eb468-fdda-4d9d-a362-b13443842254 {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:33:37.558214 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:37.574262 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-33aed2b2-bfcc-41f2-be27-5a305c3984c5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 200 len: 1332 microversion: 2.1 time: 0.126581 Aug 21 01:33:37.574752 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 482/949] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:37 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 1332 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:38.605752 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-035d1558-08e8-4e44-8018-9a922c11f00d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:38.606374 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-035d1558-08e8-4e44-8018-9a922c11f00d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:38.612986 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-035d1558-08e8-4e44-8018-9a922c11f00d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:38.613468 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-035d1558-08e8-4e44-8018-9a922c11f00d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:38.613909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-035d1558-08e8-4e44-8018-9a922c11f00d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:38.614631 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-035d1558-08e8-4e44-8018-9a922c11f00d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:38.615089 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-035d1558-08e8-4e44-8018-9a922c11f00d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:38.615517 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-035d1558-08e8-4e44-8018-9a922c11f00d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:38.629629 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-035d1558-08e8-4e44-8018-9a922c11f00d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-035d1558-08e8-4e44-8018-9a922c11f00d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 HTTP/1.1 Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:33:38.631222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:33:38.632840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:33:38.632840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:33:38.632840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:33:38.632840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:33:38.632840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:33:38.632840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:33:38.632840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:33:38.632840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:33:38.632840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:33:38.632840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:33:38.632840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:33:38.632840 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '049eb468-fdda-4d9d-a362-b13443842254'} Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:33:38.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:33:38.635036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:38.636303 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-035d1558-08e8-4e44-8018-9a922c11f00d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 404 to user: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:33:38.636303 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-035d1558-08e8-4e44-8018-9a922c11f00d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 404 len: 111 microversion: 2.1 time: 0.031276 Aug 21 01:33:38.636303 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 468/950] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:38 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:33:38.641325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-92405ed0-389e-4f1b-a3e2-790e665377d1 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '.version_select at 0x7f9da3393b80>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:38.741473 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-92405ed0-389e-4f1b-a3e2-790e665377d1 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1210986866" status: 202 len: 0 microversion: 2.1 time: 0.103283 Aug 21 01:33:38.742048 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 483/951] 10.222.0.47 () {58 vars in 1221 bytes} [Sun Aug 21 01:33:38 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1210986866 => generated 0 bytes in 105 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:33:39.635551 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f584e490-efce-477d-a91f-8b82cad77f33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:39.636745 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f584e490-efce-477d-a91f-8b82cad77f33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:39.643734 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f584e490-efce-477d-a91f-8b82cad77f33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:39.644249 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f584e490-efce-477d-a91f-8b82cad77f33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:39.644704 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f584e490-efce-477d-a91f-8b82cad77f33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:39.645384 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f584e490-efce-477d-a91f-8b82cad77f33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:39.646009 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f584e490-efce-477d-a91f-8b82cad77f33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:39.646434 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f584e490-efce-477d-a91f-8b82cad77f33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:39.736049 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-f584e490-efce-477d-a91f-8b82cad77f33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lazy-loading 'services' on Instance uuid d1a9e859-eddf-4f45-8e02-168201b29a63 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:33:40.019190 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f584e490-efce-477d-a91f-8b82cad77f33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/os-volume_attachments/fc6ceacd-e32a-4de3-bd34-144c0e60919d" status: 202 len: 0 microversion: 2.1 time: 0.387390 Aug 21 01:33:40.019719 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 469/952] 10.222.0.47 () {58 vars in 1416 bytes} [Sun Aug 21 01:33:39 2022] DELETE /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/os-volume_attachments/fc6ceacd-e32a-4de3-bd34-144c0e60919d => generated 0 bytes in 389 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:33:40.029602 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-0112e346-e10c-43e3-87e9-e64c030df486 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:40.032069 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-0112e346-e10c-43e3-87e9-e64c030df486 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:40.039404 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0112e346-e10c-43e3-87e9-e64c030df486 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:40.039992 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0112e346-e10c-43e3-87e9-e64c030df486 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:40.040376 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0112e346-e10c-43e3-87e9-e64c030df486 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:40.041090 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0112e346-e10c-43e3-87e9-e64c030df486 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:40.041547 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0112e346-e10c-43e3-87e9-e64c030df486 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:40.041986 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0112e346-e10c-43e3-87e9-e64c030df486 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:40.075345 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-0112e346-e10c-43e3-87e9-e64c030df486 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050257 Aug 21 01:33:40.075918 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 484/953] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:33:40 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:41.090463 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e6d54a0b-80dc-4097-8bec-73ea88f61729 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:41.092790 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e6d54a0b-80dc-4097-8bec-73ea88f61729 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:41.099085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e6d54a0b-80dc-4097-8bec-73ea88f61729 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:41.099569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e6d54a0b-80dc-4097-8bec-73ea88f61729 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:41.100000 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e6d54a0b-80dc-4097-8bec-73ea88f61729 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:41.100722 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e6d54a0b-80dc-4097-8bec-73ea88f61729 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:41.101804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e6d54a0b-80dc-4097-8bec-73ea88f61729 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:41.102229 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e6d54a0b-80dc-4097-8bec-73ea88f61729 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:41.131248 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e6d54a0b-80dc-4097-8bec-73ea88f61729 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046687 Aug 21 01:33:41.131797 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 470/954] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:33:41 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:42.143080 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d85910e5-8b1e-49b7-9277-d8b9094e1998 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:42.145262 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d85910e5-8b1e-49b7-9277-d8b9094e1998 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:42.155967 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d85910e5-8b1e-49b7-9277-d8b9094e1998 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:42.155967 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d85910e5-8b1e-49b7-9277-d8b9094e1998 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:42.155967 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d85910e5-8b1e-49b7-9277-d8b9094e1998 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:42.155967 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d85910e5-8b1e-49b7-9277-d8b9094e1998 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:42.156354 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d85910e5-8b1e-49b7-9277-d8b9094e1998 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:42.156354 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d85910e5-8b1e-49b7-9277-d8b9094e1998 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:42.183712 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d85910e5-8b1e-49b7-9277-d8b9094e1998 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044443 Aug 21 01:33:42.184521 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 485/955] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:33:42 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 21 01:33:42.247078 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-06332013-4ed6-44ac-b9cb-97ab138458d8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:42.248326 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-06332013-4ed6-44ac-b9cb-97ab138458d8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:42.253978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06332013-4ed6-44ac-b9cb-97ab138458d8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:42.254405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06332013-4ed6-44ac-b9cb-97ab138458d8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:42.254844 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06332013-4ed6-44ac-b9cb-97ab138458d8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:42.255534 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06332013-4ed6-44ac-b9cb-97ab138458d8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:42.255988 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06332013-4ed6-44ac-b9cb-97ab138458d8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:42.256441 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06332013-4ed6-44ac-b9cb-97ab138458d8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:42.283257 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-06332013-4ed6-44ac-b9cb-97ab138458d8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:33:42.581214 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-06332013-4ed6-44ac-b9cb-97ab138458d8 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 204 len: 0 microversion: 2.1 time: 0.337609 Aug 21 01:33:42.582057 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 471/956] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:33:42 2022] DELETE /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 21 01:33:42.591268 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ab9f9a5a-8f60-4193-8b9e-3016a2385e7d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:42.591878 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-ab9f9a5a-8f60-4193-8b9e-3016a2385e7d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:42.597940 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ab9f9a5a-8f60-4193-8b9e-3016a2385e7d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:42.598431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ab9f9a5a-8f60-4193-8b9e-3016a2385e7d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:42.598860 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ab9f9a5a-8f60-4193-8b9e-3016a2385e7d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:42.599543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ab9f9a5a-8f60-4193-8b9e-3016a2385e7d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:42.599994 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ab9f9a5a-8f60-4193-8b9e-3016a2385e7d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:42.600439 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ab9f9a5a-8f60-4193-8b9e-3016a2385e7d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:42.742471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ab9f9a5a-8f60-4193-8b9e-3016a2385e7d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:42.750459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ab9f9a5a-8f60-4193-8b9e-3016a2385e7d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1680 microversion: 2.1 time: 0.163207 Aug 21 01:33:42.752474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 486/957] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:42 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1680 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:43.772796 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-eb971614-2b58-4698-b5cb-b1514839d80e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:43.773553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-eb971614-2b58-4698-b5cb-b1514839d80e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:43.780418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb971614-2b58-4698-b5cb-b1514839d80e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:43.780900 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb971614-2b58-4698-b5cb-b1514839d80e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:43.781333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb971614-2b58-4698-b5cb-b1514839d80e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:43.782026 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb971614-2b58-4698-b5cb-b1514839d80e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:43.782476 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb971614-2b58-4698-b5cb-b1514839d80e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:43.782943 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-eb971614-2b58-4698-b5cb-b1514839d80e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:44.006831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-eb971614-2b58-4698-b5cb-b1514839d80e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:44.015189 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-eb971614-2b58-4698-b5cb-b1514839d80e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1680 microversion: 2.1 time: 0.246660 Aug 21 01:33:44.015778 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 472/958] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:43 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1680 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:33:45.037126 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-af876d32-a0fe-470a-8f61-162c60c55e9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:45.037797 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-af876d32-a0fe-470a-8f61-162c60c55e9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:45.043995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-af876d32-a0fe-470a-8f61-162c60c55e9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:45.044572 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-af876d32-a0fe-470a-8f61-162c60c55e9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:45.044975 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-af876d32-a0fe-470a-8f61-162c60c55e9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:45.045685 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-af876d32-a0fe-470a-8f61-162c60c55e9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:45.046140 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-af876d32-a0fe-470a-8f61-162c60c55e9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:45.046574 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-af876d32-a0fe-470a-8f61-162c60c55e9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:45.216912 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-af876d32-a0fe-470a-8f61-162c60c55e9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:45.227450 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-af876d32-a0fe-470a-8f61-162c60c55e9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1680 microversion: 2.1 time: 0.194039 Aug 21 01:33:45.227575 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 487/959] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:45 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1680 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:45.558273 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-51ba9607-f154-48be-b168-bea2a96e3e36 req-c0f63a18-2007-452d-a010-9d3cc2fc50d3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1a9e859-eddf-4f45-8e02-168201b29a63", "name": "network-vif-deleted", "tag": "316e8409-378a-47c3-a927-8b6aea8ed8bd"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:33:45.572618 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-51ba9607-f154-48be-b168-bea2a96e3e36 req-c0f63a18-2007-452d-a010-9d3cc2fc50d3 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:45.573105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-51ba9607-f154-48be-b168-bea2a96e3e36 req-c0f63a18-2007-452d-a010-9d3cc2fc50d3 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:45.573538 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-51ba9607-f154-48be-b168-bea2a96e3e36 req-c0f63a18-2007-452d-a010-9d3cc2fc50d3 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:45.589562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-51ba9607-f154-48be-b168-bea2a96e3e36 req-c0f63a18-2007-452d-a010-9d3cc2fc50d3 service nova] Creating event network-vif-deleted:316e8409-378a-47c3-a927-8b6aea8ed8bd for instance d1a9e859-eddf-4f45-8e02-168201b29a63 on hv-cinder-maste Aug 21 01:33:45.595841 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-51ba9607-f154-48be-b168-bea2a96e3e36 req-c0f63a18-2007-452d-a010-9d3cc2fc50d3 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041078 Aug 21 01:33:45.595841 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 473/960] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:33:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:46.248386 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:46.249211 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:46.256076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:46.256674 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:46.258570 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:46.260006 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:46.260711 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:46.261332 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:46.295860 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:46.296384 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:46.297002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:46.297732 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lazy-loading 'fault' on Instance uuid d1a9e859-eddf-4f45-8e02-168201b29a63 {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:33:46.348520 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:46.361254 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e3058871-99d3-4625-87a5-ecc932033679 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 200 len: 1331 microversion: 2.1 time: 0.116727 Aug 21 01:33:46.362089 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 488/961] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:46 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 1331 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:47.384041 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c58ccc03-e868-40c3-8360-205c92d6f479 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:47.384896 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c58ccc03-e868-40c3-8360-205c92d6f479 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:47.391293 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c58ccc03-e868-40c3-8360-205c92d6f479 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:47.391876 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c58ccc03-e868-40c3-8360-205c92d6f479 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:47.392423 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c58ccc03-e868-40c3-8360-205c92d6f479 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:47.393197 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c58ccc03-e868-40c3-8360-205c92d6f479 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:47.393661 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c58ccc03-e868-40c3-8360-205c92d6f479 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:47.394089 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c58ccc03-e868-40c3-8360-205c92d6f479 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:47.408078 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-c58ccc03-e868-40c3-8360-205c92d6f479 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c58ccc03-e868-40c3-8360-205c92d6f479 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 HTTP/1.1 Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:33:47.409714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:33:47.411424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:33:47.411424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:33:47.411424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:33:47.411424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:33:47.411424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:33:47.411424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:33:47.411424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:33:47.411424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:33:47.411424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:33:47.411424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:33:47.411424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:33:47.411424 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': 'd1a9e859-eddf-4f45-8e02-168201b29a63'} Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:33:47.412418 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:33:47.414564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:47.415928 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c58ccc03-e868-40c3-8360-205c92d6f479 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 404 to user: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:33:47.415928 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c58ccc03-e868-40c3-8360-205c92d6f479 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 404 len: 111 microversion: 2.1 time: 0.031852 Aug 21 01:33:47.415928 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 474/962] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:47 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:33:47.420813 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9953f060-25d4-490d-aba1-14f05a4efe7e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '.version_select at 0x7f9da335a5e0>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:47.512439 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9953f060-25d4-490d-aba1-14f05a4efe7e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1630789153" status: 202 len: 0 microversion: 2.1 time: 0.095450 Aug 21 01:33:47.513086 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 489/963] 10.222.0.47 () {58 vars in 1221 bytes} [Sun Aug 21 01:33:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1630789153 => generated 0 bytes in 97 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:33:48.239957 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-931954ad-d5f5-4773-ac56-8851a0bcf0f6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: .version_select at 0x7f9da339be50>, body: {"keypair": {"name": "tempest-keypair-1633583235", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB6//VLrt9JzWOiWpZl5TQIfQIHWr1mEeOJV/AzJ+raGTpEWIwq/v1pdJkD/yHKwBUaa/tECqJ/oe3JVIITv9NgVB3pQq0lK2Z2lVUtUBEXCTCgty3MZTXtdWmws5i9PjA=="}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:33:48.257666 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-931954ad-d5f5-4773-ac56-8851a0bcf0f6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for project 122b46d9e29843a88c7ca5f019426539. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:33:48.261646 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-931954ad-d5f5-4773-ac56-8851a0bcf0f6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for user a91ac0ba7e8c4418930852b0fa184247 and project 122b46d9e29843a88c7ca5f019426539. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:33:48.365514 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-931954ad-d5f5-4773-ac56-8851a0bcf0f6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.129448 Aug 21 01:33:48.366124 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 475/964] 10.222.0.47 () {60 vars in 1159 bytes} [Sun Aug 21 01:33:48 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:50.652968 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-98612555", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "99061e98-94a6-4cdf-b3c1-c9af69a6346c"}], "security_groups": [{"name": "tempest-securitygroup--1320778319"}], "key_name": "tempest-keypair-1633583235", "user_data": "}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:33:50.764633 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.network.neutron [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] validate_networks() for [('99061e98-94a6-4cdf-b3c1-c9af69a6346c', None, None, None, None, None)] {{(pid=121277) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:33:51.428448 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Flavor limits 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:33:51.428879 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Image limits 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:33:51.429356 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Flavor pref 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:33:51.430115 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Image pref 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:33:51.430317 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:33:51.439412 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:51.439901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:51.440370 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:51.441108 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:51.441563 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:51.441989 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:51.461074 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for project 122b46d9e29843a88c7ca5f019426539. Resources: {'instances', 'ram', 'cores'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:33:51.467712 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for user a91ac0ba7e8c4418930852b0fa184247 and project 122b46d9e29843a88c7ca5f019426539. Resources: {'instances', 'ram', 'cores'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:33:51.481076 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Going to run 1 instances... {{(pid=121277) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:33:51.483406 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121277) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:33:51.609600 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-1f270b7e-979a-4cd8-95c6-a64ef4e8ebd5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.960938 Aug 21 01:33:51.610246 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 490/965] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:33:50 2022] POST /compute/v2.1/servers => generated 402 bytes in 962 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:33:51.639110 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-0292351c-acda-4f89-92f4-6af0f32bf5d5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:51.640273 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-0292351c-acda-4f89-92f4-6af0f32bf5d5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:51.701449 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0292351c-acda-4f89-92f4-6af0f32bf5d5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:51.713074 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-0292351c-acda-4f89-92f4-6af0f32bf5d5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1240 microversion: 2.1 time: 0.089543 Aug 21 01:33:51.716079 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 476/966] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:51 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1240 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:52.736711 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-ebb454bc-7b16-423e-a152-2f790e2d1a28 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:52.737852 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-ebb454bc-7b16-423e-a152-2f790e2d1a28 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:52.744204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ebb454bc-7b16-423e-a152-2f790e2d1a28 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:52.744913 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ebb454bc-7b16-423e-a152-2f790e2d1a28 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:52.745520 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ebb454bc-7b16-423e-a152-2f790e2d1a28 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:52.746390 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ebb454bc-7b16-423e-a152-2f790e2d1a28 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:52.747207 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ebb454bc-7b16-423e-a152-2f790e2d1a28 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:52.747799 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-ebb454bc-7b16-423e-a152-2f790e2d1a28 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:52.826072 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-ebb454bc-7b16-423e-a152-2f790e2d1a28 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:52.834344 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-ebb454bc-7b16-423e-a152-2f790e2d1a28 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1244 microversion: 2.1 time: 0.101949 Aug 21 01:33:52.834906 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 491/967] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:52 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1244 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:53.349476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-5f3b3ead-b0e9-4e7c-bc43-1c7045f71684 req-102c729b-e24b-4551-a7a7-3a689440bcf8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "119204ec-e0f8-439e-b985-b934d251684f", "tag": "bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:33:53.357022 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c06750c8-5ffd-4521-bc9f-b383b19513a5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:53.357530 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c06750c8-5ffd-4521-bc9f-b383b19513a5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:53.367012 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c06750c8-5ffd-4521-bc9f-b383b19513a5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:53.367474 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c06750c8-5ffd-4521-bc9f-b383b19513a5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:53.367904 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c06750c8-5ffd-4521-bc9f-b383b19513a5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:53.368630 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c06750c8-5ffd-4521-bc9f-b383b19513a5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:53.369087 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c06750c8-5ffd-4521-bc9f-b383b19513a5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:53.369512 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c06750c8-5ffd-4521-bc9f-b383b19513a5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:53.378692 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-5f3b3ead-b0e9-4e7c-bc43-1c7045f71684 req-102c729b-e24b-4551-a7a7-3a689440bcf8 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:53.379311 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-5f3b3ead-b0e9-4e7c-bc43-1c7045f71684 req-102c729b-e24b-4551-a7a7-3a689440bcf8 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:53.379850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-5f3b3ead-b0e9-4e7c-bc43-1c7045f71684 req-102c729b-e24b-4551-a7a7-3a689440bcf8 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:53.394249 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-5f3b3ead-b0e9-4e7c-bc43-1c7045f71684 req-102c729b-e24b-4551-a7a7-3a689440bcf8 service nova] Creating event network-changed:bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f for instance 119204ec-e0f8-439e-b985-b934d251684f on hv-cinder-maste Aug 21 01:33:53.397471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c06750c8-5ffd-4521-bc9f-b383b19513a5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:33:53.399863 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-5f3b3ead-b0e9-4e7c-bc43-1c7045f71684 req-102c729b-e24b-4551-a7a7-3a689440bcf8 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053923 Aug 21 01:33:53.406669 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 492/968] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:33:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:53.786917 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c06750c8-5ffd-4521-bc9f-b383b19513a5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "DELETE /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 204 len: 0 microversion: 2.1 time: 0.444400 Aug 21 01:33:53.791096 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 477/969] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:33:53 2022] DELETE /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 0 bytes in 446 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:33:53.798571 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-31bb6728-4db9-43ae-a647-359b1ebd1bc6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:53.799719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-31bb6728-4db9-43ae-a647-359b1ebd1bc6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:53.808242 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31bb6728-4db9-43ae-a647-359b1ebd1bc6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:53.808881 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31bb6728-4db9-43ae-a647-359b1ebd1bc6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:53.809559 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31bb6728-4db9-43ae-a647-359b1ebd1bc6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:53.810916 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31bb6728-4db9-43ae-a647-359b1ebd1bc6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:53.811449 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31bb6728-4db9-43ae-a647-359b1ebd1bc6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:53.812039 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31bb6728-4db9-43ae-a647-359b1ebd1bc6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:53.857089 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3293cbdd-a88a-4ff6-91d6-221d0d2dc296 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:53.858081 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3293cbdd-a88a-4ff6-91d6-221d0d2dc296 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:53.864762 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3293cbdd-a88a-4ff6-91d6-221d0d2dc296 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:53.865406 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3293cbdd-a88a-4ff6-91d6-221d0d2dc296 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:53.865952 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3293cbdd-a88a-4ff6-91d6-221d0d2dc296 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:53.867363 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3293cbdd-a88a-4ff6-91d6-221d0d2dc296 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:53.867929 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3293cbdd-a88a-4ff6-91d6-221d0d2dc296 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:53.868498 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3293cbdd-a88a-4ff6-91d6-221d0d2dc296 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:53.961076 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3293cbdd-a88a-4ff6-91d6-221d0d2dc296 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:53.971007 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3293cbdd-a88a-4ff6-91d6-221d0d2dc296 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1300 microversion: 2.1 time: 0.117953 Aug 21 01:33:53.971466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 478/970] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:53 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1300 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:33:54.002477 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-31bb6728-4db9-43ae-a647-359b1ebd1bc6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:54.010954 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-31bb6728-4db9-43ae-a647-359b1ebd1bc6 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.217469 Aug 21 01:33:54.011543 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 493/971] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:53 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:54.992072 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4564fc9b-6b83-4c3a-8263-8a11c71b21c7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:54.992748 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4564fc9b-6b83-4c3a-8263-8a11c71b21c7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:54.999530 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4564fc9b-6b83-4c3a-8263-8a11c71b21c7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:55.000020 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4564fc9b-6b83-4c3a-8263-8a11c71b21c7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:55.000499 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4564fc9b-6b83-4c3a-8263-8a11c71b21c7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:55.001214 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4564fc9b-6b83-4c3a-8263-8a11c71b21c7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:55.001673 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4564fc9b-6b83-4c3a-8263-8a11c71b21c7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:55.002101 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4564fc9b-6b83-4c3a-8263-8a11c71b21c7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:55.032776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4468e413-a9d6-493c-bda8-bf518b63c6cf tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:55.033467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4468e413-a9d6-493c-bda8-bf518b63c6cf tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:55.041969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4468e413-a9d6-493c-bda8-bf518b63c6cf tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:55.042456 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4468e413-a9d6-493c-bda8-bf518b63c6cf tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:55.042899 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4468e413-a9d6-493c-bda8-bf518b63c6cf tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:55.043601 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4468e413-a9d6-493c-bda8-bf518b63c6cf tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:55.044093 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4468e413-a9d6-493c-bda8-bf518b63c6cf tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:55.044526 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4468e413-a9d6-493c-bda8-bf518b63c6cf tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:55.069257 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4564fc9b-6b83-4c3a-8263-8a11c71b21c7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:55.073386 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=3,nova_cell1:INSERT=4 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:33:55.078005 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4564fc9b-6b83-4c3a-8263-8a11c71b21c7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1300 microversion: 2.1 time: 0.089953 Aug 21 01:33:55.078572 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 479/972] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:54 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1300 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:55.231066 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4468e413-a9d6-493c-bda8-bf518b63c6cf tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:55.239244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4468e413-a9d6-493c-bda8-bf518b63c6cf tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1568 microversion: 2.1 time: 0.210333 Aug 21 01:33:55.239815 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 494/973] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:55 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1568 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:33:56.104311 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-36a0b3f4-f459-423f-9282-a22060f0854f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:56.105758 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-36a0b3f4-f459-423f-9282-a22060f0854f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:56.118605 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-36a0b3f4-f459-423f-9282-a22060f0854f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:56.119114 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-36a0b3f4-f459-423f-9282-a22060f0854f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:56.119616 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-36a0b3f4-f459-423f-9282-a22060f0854f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:56.120405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-36a0b3f4-f459-423f-9282-a22060f0854f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:56.120867 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-36a0b3f4-f459-423f-9282-a22060f0854f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:56.122004 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-36a0b3f4-f459-423f-9282-a22060f0854f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:56.215262 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-36a0b3f4-f459-423f-9282-a22060f0854f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:56.232092 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-36a0b3f4-f459-423f-9282-a22060f0854f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1298 microversion: 2.1 time: 0.132425 Aug 21 01:33:56.239385 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 480/974] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:56 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1298 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:56.261979 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8d712ab8-cbd6-4c02-a1eb-d7d18d697e20 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:56.262777 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8d712ab8-cbd6-4c02-a1eb-d7d18d697e20 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:56.269645 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d712ab8-cbd6-4c02-a1eb-d7d18d697e20 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:56.270124 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d712ab8-cbd6-4c02-a1eb-d7d18d697e20 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:56.271113 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d712ab8-cbd6-4c02-a1eb-d7d18d697e20 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:56.271240 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d712ab8-cbd6-4c02-a1eb-d7d18d697e20 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:56.271694 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d712ab8-cbd6-4c02-a1eb-d7d18d697e20 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:56.272136 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d712ab8-cbd6-4c02-a1eb-d7d18d697e20 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:56.577494 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8d712ab8-cbd6-4c02-a1eb-d7d18d697e20 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:56.584984 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8d712ab8-cbd6-4c02-a1eb-d7d18d697e20 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1450 microversion: 2.1 time: 0.327659 Aug 21 01:33:56.585878 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 495/975] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:56 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1450 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:56.829970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-87e36481-5cf4-4795-a0a7-bb5add6a2402 req-6bb4c646-b6c4-46a3-9442-34c1970d7e62 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "119204ec-e0f8-439e-b985-b934d251684f", "name": "network-vif-deleted", "tag": "bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:33:56.840642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-87e36481-5cf4-4795-a0a7-bb5add6a2402 req-6bb4c646-b6c4-46a3-9442-34c1970d7e62 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:56.841202 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-87e36481-5cf4-4795-a0a7-bb5add6a2402 req-6bb4c646-b6c4-46a3-9442-34c1970d7e62 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:56.841645 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-87e36481-5cf4-4795-a0a7-bb5add6a2402 req-6bb4c646-b6c4-46a3-9442-34c1970d7e62 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:56.856466 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-87e36481-5cf4-4795-a0a7-bb5add6a2402 req-6bb4c646-b6c4-46a3-9442-34c1970d7e62 service nova] Creating event network-vif-deleted:bdd2fff3-c630-4a19-ac51-fd8cd3d3dd5f for instance 119204ec-e0f8-439e-b985-b934d251684f on hv-cinder-maste Aug 21 01:33:56.861314 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-87e36481-5cf4-4795-a0a7-bb5add6a2402 req-6bb4c646-b6c4-46a3-9442-34c1970d7e62 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035100 Aug 21 01:33:56.861918 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 481/976] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:33:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:57.266073 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-50448412-4775-482a-ae23-07a8fee821a6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:57.266687 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-50448412-4775-482a-ae23-07a8fee821a6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:57.273681 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50448412-4775-482a-ae23-07a8fee821a6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:57.274433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50448412-4775-482a-ae23-07a8fee821a6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:57.274881 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50448412-4775-482a-ae23-07a8fee821a6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:57.275590 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50448412-4775-482a-ae23-07a8fee821a6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:57.276050 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50448412-4775-482a-ae23-07a8fee821a6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:57.276509 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-50448412-4775-482a-ae23-07a8fee821a6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:57.470022 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-50448412-4775-482a-ae23-07a8fee821a6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:57.478579 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-50448412-4775-482a-ae23-07a8fee821a6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1366 microversion: 2.1 time: 0.216233 Aug 21 01:33:57.479237 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 496/977] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:57 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1366 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:57.607723 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fab21da6-ad37-4390-8c53-656adce946f5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:57.608511 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fab21da6-ad37-4390-8c53-656adce946f5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:57.619789 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fab21da6-ad37-4390-8c53-656adce946f5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:57.620326 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fab21da6-ad37-4390-8c53-656adce946f5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:57.620768 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fab21da6-ad37-4390-8c53-656adce946f5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:57.621481 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fab21da6-ad37-4390-8c53-656adce946f5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:57.621936 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fab21da6-ad37-4390-8c53-656adce946f5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:57.622365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fab21da6-ad37-4390-8c53-656adce946f5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:57.697312 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-fab21da6-ad37-4390-8c53-656adce946f5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '03d34d1e66404d259599806d040628ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ac06ecc6fc93477ba4909284525df514', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:57.705632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fab21da6-ad37-4390-8c53-656adce946f5 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 200 len: 1219 microversion: 2.1 time: 0.102507 Aug 21 01:33:57.706230 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 482/978] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:57 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 1219 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:58.499873 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f3c8d9e7-e98e-42b1-8edd-55a0b248cd6c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:58.501202 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f3c8d9e7-e98e-42b1-8edd-55a0b248cd6c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:58.508406 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f3c8d9e7-e98e-42b1-8edd-55a0b248cd6c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:58.509098 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f3c8d9e7-e98e-42b1-8edd-55a0b248cd6c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:58.509643 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f3c8d9e7-e98e-42b1-8edd-55a0b248cd6c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:58.510467 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f3c8d9e7-e98e-42b1-8edd-55a0b248cd6c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:58.511832 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f3c8d9e7-e98e-42b1-8edd-55a0b248cd6c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:58.512407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f3c8d9e7-e98e-42b1-8edd-55a0b248cd6c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:58.687418 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f3c8d9e7-e98e-42b1-8edd-55a0b248cd6c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:58.696080 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f3c8d9e7-e98e-42b1-8edd-55a0b248cd6c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1366 microversion: 2.1 time: 0.200708 Aug 21 01:33:58.696696 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 497/979] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:58 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1366 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:33:58.727392 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8a4deeb9-5c3b-419a-9454-846828d1929c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:58.731419 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8a4deeb9-5c3b-419a-9454-846828d1929c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] [instance: 119204ec-e0f8-439e-b985-b934d251684f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:58.734874 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a4deeb9-5c3b-419a-9454-846828d1929c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:58.735366 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a4deeb9-5c3b-419a-9454-846828d1929c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:58.735811 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a4deeb9-5c3b-419a-9454-846828d1929c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:58.736544 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a4deeb9-5c3b-419a-9454-846828d1929c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:58.737007 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a4deeb9-5c3b-419a-9454-846828d1929c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:58.737531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8a4deeb9-5c3b-419a-9454-846828d1929c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-8a4deeb9-5c3b-419a-9454-846828d1929c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] HTTP exception thrown: Instance 119204ec-e0f8-439e-b985-b934d251684f could not be found. Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8a4deeb9-5c3b-419a-9454-846828d1929c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Request method failure captured: Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f HTTP/1.1 Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYo0TFwy1WhpQYCjlgay18ASwPjqPj7mdnaUqE5Fi3xya4UmE9LSzUkY-Cr7gAiHXucyUt8rQ8UwCVXOqqhZ0S8buL-Cv3BJJwh-GiFI3bbln6m0jT5860LJIKwRGMoAhFSnv2OSEEGGLDb1SbZsUzbYvgE9uQ54rREV4GSL8tccAN4 Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: ac06ecc6fc93477ba4909284525df514 Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-TestShelveInstance-132004812 Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:33:58.754714 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:33:58.758676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/ac06ecc6fc93477ba4909284525df514"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_ac06ecc6fc93477ba4909284525df514"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ac06ecc6fc93477ba4909284525df514"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/ac06ecc6fc93477ba4909284525df514"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:33:58.758676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-TestShelveInstance-132004812 Aug 21 01:33:58.758676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: ac06ecc6fc93477ba4909284525df514 Aug 21 01:33:58.758676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-TestShelveInstance-132004812 Aug 21 01:33:58.758676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-TestShelveInstance-132004812-project Aug 21 01:33:58.758676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:33:58.758676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:33:58.758676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: 03d34d1e66404d259599806d040628ef Aug 21 01:33:58.758676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-TestShelveInstance-132004812-project Aug 21 01:33:58.758676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:33:58.758676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 119204ec-e0f8-439e-b985-b934d251684f could not be found. Aug 21 01:33:58.758676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '119204ec-e0f8-439e-b985-b934d251684f'} Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 119204ec-e0f8-439e-b985-b934d251684f could not be found. Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:33:58.759642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 119204ec-e0f8-439e-b985-b934d251684f could not be found. Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 119204ec-e0f8-439e-b985-b934d251684f could not be found. Aug 21 01:33:58.762771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:33:58.764451 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8a4deeb9-5c3b-419a-9454-846828d1929c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Returning 404 to user: Instance 119204ec-e0f8-439e-b985-b934d251684f could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:33:58.764451 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8a4deeb9-5c3b-419a-9454-846828d1929c tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f" status: 404 len: 111 microversion: 2.1 time: 0.032618 Aug 21 01:33:58.764451 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 483/980] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:58 2022] GET /compute/v2.1/servers/119204ec-e0f8-439e-b985-b934d251684f => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:33:58.880578 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-061c48dc-acf5-4a7e-9373-117103c61635 req-4835f28a-aa0e-40f7-be81-fb4de71511aa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8554ae19-faa1-4d72-8d84-529daa809281", "tag": "7d30bbc9-a97b-43d2-b319-9d4924803d0f"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:33:58.890953 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-061c48dc-acf5-4a7e-9373-117103c61635 req-4835f28a-aa0e-40f7-be81-fb4de71511aa service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:58.891442 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-061c48dc-acf5-4a7e-9373-117103c61635 req-4835f28a-aa0e-40f7-be81-fb4de71511aa service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:58.891882 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-061c48dc-acf5-4a7e-9373-117103c61635 req-4835f28a-aa0e-40f7-be81-fb4de71511aa service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:58.896057 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=100 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:33:58.963445 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-061c48dc-acf5-4a7e-9373-117103c61635 req-4835f28a-aa0e-40f7-be81-fb4de71511aa service nova] Creating event network-changed:7d30bbc9-a97b-43d2-b319-9d4924803d0f for instance 8554ae19-faa1-4d72-8d84-529daa809281 on hv-cinder-maste Aug 21 01:33:58.968655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-061c48dc-acf5-4a7e-9373-117103c61635 req-4835f28a-aa0e-40f7-be81-fb4de71511aa service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.092658 Aug 21 01:33:58.969258 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 498/981] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:33:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:33:59.034390 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-674cbc76-273d-4297-91b1-555720b911f0 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] Calling method '.version_select at 0x7f9da33bd310>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:59.158841 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-674cbc76-273d-4297-91b1-555720b911f0 tempest-TestShelveInstance-132004812 tempest-TestShelveInstance-132004812-project] 10.222.0.47 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-281379194" status: 202 len: 0 microversion: 2.1 time: 0.129704 Aug 21 01:33:59.159431 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 484/982] 10.222.0.47 () {58 vars in 1251 bytes} [Sun Aug 21 01:33:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-281379194 => generated 0 bytes in 132 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:33:59.723504 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4052bb8e-1434-4b70-ad50-ad98e7918e42 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:33:59.725028 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4052bb8e-1434-4b70-ad50-ad98e7918e42 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:33:59.732176 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4052bb8e-1434-4b70-ad50-ad98e7918e42 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:59.732913 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4052bb8e-1434-4b70-ad50-ad98e7918e42 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:59.733510 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4052bb8e-1434-4b70-ad50-ad98e7918e42 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:59.734438 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4052bb8e-1434-4b70-ad50-ad98e7918e42 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:33:59.735685 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4052bb8e-1434-4b70-ad50-ad98e7918e42 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:33:59.736308 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4052bb8e-1434-4b70-ad50-ad98e7918e42 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:33:59.969880 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4052bb8e-1434-4b70-ad50-ad98e7918e42 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:33:59.979572 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4052bb8e-1434-4b70-ad50-ad98e7918e42 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1366 microversion: 2.1 time: 0.261283 Aug 21 01:33:59.980210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 499/983] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:33:59 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1366 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:34:01.001608 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-5d2e3b8a-6f86-422b-bc15-669c9340e1f5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:01.002884 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-5d2e3b8a-6f86-422b-bc15-669c9340e1f5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:01.010849 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d2e3b8a-6f86-422b-bc15-669c9340e1f5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:01.011410 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d2e3b8a-6f86-422b-bc15-669c9340e1f5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:01.012083 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d2e3b8a-6f86-422b-bc15-669c9340e1f5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:01.013034 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d2e3b8a-6f86-422b-bc15-669c9340e1f5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:01.013607 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d2e3b8a-6f86-422b-bc15-669c9340e1f5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:01.014107 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d2e3b8a-6f86-422b-bc15-669c9340e1f5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:01.264497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5d2e3b8a-6f86-422b-bc15-669c9340e1f5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:34:01.274893 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5d2e3b8a-6f86-422b-bc15-669c9340e1f5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1366 microversion: 2.1 time: 0.278152 Aug 21 01:34:01.276865 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 485/984] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:00 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1366 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:34:02.284953 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-87214619-4a87-4b94-a9b4-9aa88af55842 req-41d4b1d4-8c96-4556-9f0d-558d64620703 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8554ae19-faa1-4d72-8d84-529daa809281", "name": "network-vif-plugged", "status": "completed", "tag": "7d30bbc9-a97b-43d2-b319-9d4924803d0f"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:34:02.287472 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:34:02.289404 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=6 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:34:02.295220 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-87214619-4a87-4b94-a9b4-9aa88af55842 req-41d4b1d4-8c96-4556-9f0d-558d64620703 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:02.295991 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-87214619-4a87-4b94-a9b4-9aa88af55842 req-41d4b1d4-8c96-4556-9f0d-558d64620703 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:02.296631 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-87214619-4a87-4b94-a9b4-9aa88af55842 req-41d4b1d4-8c96-4556-9f0d-558d64620703 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:02.297937 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2862595d-b2db-4ad1-bfcd-8e0e06f0b235 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:02.298547 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2862595d-b2db-4ad1-bfcd-8e0e06f0b235 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:02.301496 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=6 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:34:02.306886 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2862595d-b2db-4ad1-bfcd-8e0e06f0b235 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:02.307382 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2862595d-b2db-4ad1-bfcd-8e0e06f0b235 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:02.307819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2862595d-b2db-4ad1-bfcd-8e0e06f0b235 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:02.308549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2862595d-b2db-4ad1-bfcd-8e0e06f0b235 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:02.310768 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2862595d-b2db-4ad1-bfcd-8e0e06f0b235 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:02.310903 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-87214619-4a87-4b94-a9b4-9aa88af55842 req-41d4b1d4-8c96-4556-9f0d-558d64620703 service nova] Creating event network-vif-plugged:7d30bbc9-a97b-43d2-b319-9d4924803d0f for instance 8554ae19-faa1-4d72-8d84-529daa809281 on hv-cinder-maste Aug 21 01:34:02.311207 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2862595d-b2db-4ad1-bfcd-8e0e06f0b235 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:02.315663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-87214619-4a87-4b94-a9b4-9aa88af55842 req-41d4b1d4-8c96-4556-9f0d-558d64620703 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044864 Aug 21 01:34:02.316305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 500/985] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:34:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:34:02.494627 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2862595d-b2db-4ad1-bfcd-8e0e06f0b235 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:34:02.502403 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2862595d-b2db-4ad1-bfcd-8e0e06f0b235 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1366 microversion: 2.1 time: 0.208284 Aug 21 01:34:02.502976 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 486/986] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:02 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1366 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:34:03.533294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3ade313c-dfe7-4fc5-87e2-004ae303234c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:03.534613 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3ade313c-dfe7-4fc5-87e2-004ae303234c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:03.537477 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=64,nova_api:DELETE=2,nova_api:INSERT=3 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:34:03.549605 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3ade313c-dfe7-4fc5-87e2-004ae303234c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:03.550334 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3ade313c-dfe7-4fc5-87e2-004ae303234c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:03.550989 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3ade313c-dfe7-4fc5-87e2-004ae303234c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:03.551929 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3ade313c-dfe7-4fc5-87e2-004ae303234c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:03.552611 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3ade313c-dfe7-4fc5-87e2-004ae303234c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:03.553242 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3ade313c-dfe7-4fc5-87e2-004ae303234c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:03.702981 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3ade313c-dfe7-4fc5-87e2-004ae303234c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:34:03.707269 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3ade313c-dfe7-4fc5-87e2-004ae303234c tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1366 microversion: 2.1 time: 0.182379 Aug 21 01:34:03.708137 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 501/987] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:03 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1366 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:34:04.729933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c10d8e09-c43d-429a-9867-5a3a25fe4ad7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:04.731296 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c10d8e09-c43d-429a-9867-5a3a25fe4ad7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:04.734123 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=74,nova_api:UPDATE=3,nova_api:INSERT=1,nova_api:DELETE=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:34:04.743586 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c10d8e09-c43d-429a-9867-5a3a25fe4ad7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:04.744557 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c10d8e09-c43d-429a-9867-5a3a25fe4ad7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:04.748620 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c10d8e09-c43d-429a-9867-5a3a25fe4ad7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:04.748620 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c10d8e09-c43d-429a-9867-5a3a25fe4ad7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:04.748620 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c10d8e09-c43d-429a-9867-5a3a25fe4ad7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:04.748620 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c10d8e09-c43d-429a-9867-5a3a25fe4ad7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:04.908366 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c10d8e09-c43d-429a-9867-5a3a25fe4ad7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:34:04.918172 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c10d8e09-c43d-429a-9867-5a3a25fe4ad7 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1555 microversion: 2.1 time: 0.194122 Aug 21 01:34:04.920690 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 487/988] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:04 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1555 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:34:04.938538 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-664534f3-fb78-4636-baad-5919092ddbc0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:04.939192 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-664534f3-fb78-4636-baad-5919092ddbc0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:04.946885 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-664534f3-fb78-4636-baad-5919092ddbc0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:04.947378 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-664534f3-fb78-4636-baad-5919092ddbc0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:04.947816 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-664534f3-fb78-4636-baad-5919092ddbc0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:04.949677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-664534f3-fb78-4636-baad-5919092ddbc0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:04.949677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-664534f3-fb78-4636-baad-5919092ddbc0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:04.950514 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-664534f3-fb78-4636-baad-5919092ddbc0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:05.032423 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-664534f3-fb78-4636-baad-5919092ddbc0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.097986 Aug 21 01:34:05.033109 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 502/989] 10.222.0.47 () {58 vars in 1275 bytes} [Sun Aug 21 01:34:04 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-interface => generated 284 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:34:06.860697 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-41e1fed8-60cd-4fe0-aa63-f2362a05e5c5 req-fa69a648-147f-4175-88ef-213d27df4ec2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8554ae19-faa1-4d72-8d84-529daa809281", "tag": "7d30bbc9-a97b-43d2-b319-9d4924803d0f"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:34:06.880469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-41e1fed8-60cd-4fe0-aa63-f2362a05e5c5 req-fa69a648-147f-4175-88ef-213d27df4ec2 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:06.882473 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-41e1fed8-60cd-4fe0-aa63-f2362a05e5c5 req-fa69a648-147f-4175-88ef-213d27df4ec2 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:06.884374 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-41e1fed8-60cd-4fe0-aa63-f2362a05e5c5 req-fa69a648-147f-4175-88ef-213d27df4ec2 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:06.905399 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-41e1fed8-60cd-4fe0-aa63-f2362a05e5c5 req-fa69a648-147f-4175-88ef-213d27df4ec2 service nova] Creating event network-changed:7d30bbc9-a97b-43d2-b319-9d4924803d0f for instance 8554ae19-faa1-4d72-8d84-529daa809281 on hv-cinder-maste Aug 21 01:34:06.912166 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-41e1fed8-60cd-4fe0-aa63-f2362a05e5c5 req-fa69a648-147f-4175-88ef-213d27df4ec2 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062311 Aug 21 01:34:06.913566 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 488/990] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:34:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:34:30.477984 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=18 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:34:30.719421 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-51c9e1d3-a61f-415c-9d94-de7812eb6a35 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Action: 'create', calling method: .version_select at 0x7f9da31a9430>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1465137419", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLwX2nibYIlfC7/bi0rOrXdDxUn2ubrxjGY8vLchqvUN4NfsgDgny6IICRhkEnpK/tjpnM4EBNL49jueakhhheJAh50z374KfjrcCqzeuu7GBwJsQ32F6r2MwbulzFQWRQ=="}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:34:30.735899 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-51c9e1d3-a61f-415c-9d94-de7812eb6a35 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Getting quotas for project 4124766f4f2843229059dbbdf7a2de74. Resources: {'key_pairs'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:34:30.739835 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-51c9e1d3-a61f-415c-9d94-de7812eb6a35 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Getting quotas for user fef6461d7ca14cab829412ce3b53852a and project 4124766f4f2843229059dbbdf7a2de74. Resources: {'key_pairs'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:34:30.823190 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-51c9e1d3-a61f-415c-9d94-de7812eb6a35 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.346541 Aug 21 01:34:30.823896 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 503/991] 10.222.0.47 () {60 vars in 1159 bytes} [Sun Aug 21 01:34:30 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:34:43.519531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c93124c7-8e63-44ce-8021-fa336a1dcaf9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:43.520156 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c93124c7-8e63-44ce-8021-fa336a1dcaf9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:43.522415 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=36 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:34:43.528183 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c93124c7-8e63-44ce-8021-fa336a1dcaf9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:43.528668 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c93124c7-8e63-44ce-8021-fa336a1dcaf9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:43.529098 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c93124c7-8e63-44ce-8021-fa336a1dcaf9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:43.529788 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c93124c7-8e63-44ce-8021-fa336a1dcaf9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:43.530236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c93124c7-8e63-44ce-8021-fa336a1dcaf9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:43.530658 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c93124c7-8e63-44ce-8021-fa336a1dcaf9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:43.556196 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c93124c7-8e63-44ce-8021-fa336a1dcaf9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/ips" status: 200 len: 148 microversion: 2.1 time: 0.041486 Aug 21 01:34:43.556700 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 489/992] 10.222.0.47 () {58 vars in 1248 bytes} [Sun Aug 21 01:34:43 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/ips => generated 148 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:34:45.243250 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-0d225aec-961c-45d8-952f-4c7f9c615450 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7d846d2e-b601-446e-9d5e-412a948137bc"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:34:45.245579 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-0d225aec-961c-45d8-952f-4c7f9c615450 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:45.247738 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=4 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:34:45.248418 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=14,nova_api:INSERT=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:34:45.263915 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d225aec-961c-45d8-952f-4c7f9c615450 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:45.263915 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d225aec-961c-45d8-952f-4c7f9c615450 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:45.263915 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d225aec-961c-45d8-952f-4c7f9c615450 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:45.263915 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d225aec-961c-45d8-952f-4c7f9c615450 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:45.264361 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d225aec-961c-45d8-952f-4c7f9c615450 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:45.264361 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-0d225aec-961c-45d8-952f-4c7f9c615450 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:46.002073 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-0d225aec-961c-45d8-952f-4c7f9c615450 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.762655 Aug 21 01:34:46.002707 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 504/993] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:34:45 2022] POST /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments => generated 194 bytes in 764 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:34:51.508510 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-99269132", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1465137419", "security_groups": [{"name": "tempest-secgroup-smoke-1796196116"}], "block_device_mapping_v2": [{"uuid": "b1308b3c-c495-472c-b9cb-85570057d482", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "e4ca6005-16bf-4430-905e-1ccbd3a7f5d3"}]}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:34:51.625535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] validate_networks() for [('e4ca6005-16bf-4430-905e-1ccbd3a7f5d3', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:34:51.992757 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:34:51.993186 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:34:51.993662 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:34:51.994073 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:34:51.994484 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:34:52.003458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:52.003934 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:52.004405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:52.005133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:52.005590 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:52.006017 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:52.023027 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Getting quotas for project 4124766f4f2843229059dbbdf7a2de74. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:34:52.027817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Getting quotas for user fef6461d7ca14cab829412ce3b53852a and project 4124766f4f2843229059dbbdf7a2de74. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:34:52.085030 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:34:52.088289 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b1308b3c-c495-472c-b9cb-85570057d482',volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:34:52.599996 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-56d1087b-d1a4-4e07-a2a2-9b8b2863b899 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.095257 Aug 21 01:34:52.600797 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 490/994] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:34:51 2022] POST /compute/v2.1/servers => generated 406 bytes in 1097 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:34:52.612495 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c5554354-7877-4601-87f8-25180f6ca74b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:52.613339 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c5554354-7877-4601-87f8-25180f6ca74b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:52.670784 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c5554354-7877-4601-87f8-25180f6ca74b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:34:52.678017 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c5554354-7877-4601-87f8-25180f6ca74b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1093 microversion: 2.1 time: 0.070119 Aug 21 01:34:52.678500 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 505/995] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:52 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1093 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:34:53.710236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-615ebb18-9ecd-4ce1-9607-77302f57dbd1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:53.715380 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-615ebb18-9ecd-4ce1-9607-77302f57dbd1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:53.765572 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-615ebb18-9ecd-4ce1-9607-77302f57dbd1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:34:53.772819 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-615ebb18-9ecd-4ce1-9607-77302f57dbd1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1093 microversion: 2.1 time: 0.066551 Aug 21 01:34:53.773489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 491/996] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:53 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1093 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:34:54.604526 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-035d6678-fc96-485b-8e81-16c0bdbc8714 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7d846d2e-b601-446e-9d5e-412a948137bc"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:34:54.606892 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-035d6678-fc96-485b-8e81-16c0bdbc8714 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:54.614753 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-035d6678-fc96-485b-8e81-16c0bdbc8714 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:54.615256 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-035d6678-fc96-485b-8e81-16c0bdbc8714 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:54.615679 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-035d6678-fc96-485b-8e81-16c0bdbc8714 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:54.616427 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-035d6678-fc96-485b-8e81-16c0bdbc8714 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:54.616918 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-035d6678-fc96-485b-8e81-16c0bdbc8714 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:54.617351 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-035d6678-fc96-485b-8e81-16c0bdbc8714 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:54.698289 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-035d6678-fc96-485b-8e81-16c0bdbc8714 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Invalid volume: volume 7d846d2e-b601-446e-9d5e-412a948137bc is already attached to instances: 8554ae19-faa1-4d72-8d84-529daa809281 Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-035d6678-fc96-485b-8e81-16c0bdbc8714 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: POST /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments HTTP/1.1 Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 74 Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:34:54.700083 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:34:54.702983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:34:54.702983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:34:54.702983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:34:54.702983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:34:54.702983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:34:54.702983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:34:54.702983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:34:54.702983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:34:54.702983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:34:54.702983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Aug 21 01:34:54.702983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {"volumeAttachment": {"volumeId": "7d846d2e-b601-446e-9d5e-412a948137bc"}} Aug 21 01:34:54.702983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Invalid volume: volume 7d846d2e-b601-446e-9d5e-412a948137bc is already attached to instances: 8554ae19-faa1-4d72-8d84-529daa809281 Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'server_id': '8554ae19-faa1-4d72-8d84-529daa809281', 'body': {'volumeAttachment': {'volumeId': '7d846d2e-b601-446e-9d5e-412a948137bc'}}} Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5113, in attach_volume Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4953, in _check_volume_already_attached Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 7d846d2e-b601-446e-9d5e-412a948137bc is already attached to instances: 8554ae19-faa1-4d72-8d84-529daa809281 Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:34:54.703947 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 7d846d2e-b601-446e-9d5e-412a948137bc is already attached to instances: 8554ae19-faa1-4d72-8d84-529daa809281 Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:34:54.705333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:34:54.706620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:34:54.706620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:34:54.706620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:34:54.706620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 7d846d2e-b601-446e-9d5e-412a948137bc is already attached to instances: 8554ae19-faa1-4d72-8d84-529daa809281 Aug 21 01:34:54.706620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:34:54.706620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-035d6678-fc96-485b-8e81-16c0bdbc8714 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 400 to user: Invalid volume: volume 7d846d2e-b601-446e-9d5e-412a948137bc is already attached to instances: 8554ae19-faa1-4d72-8d84-529daa809281 {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:34:54.706620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-035d6678-fc96-485b-8e81-16c0bdbc8714 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.101889 Aug 21 01:34:54.706620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 506/997] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:34:54 2022] POST /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments => generated 174 bytes in 103 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 21 01:34:54.770966 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-b4763bd9-872f-471a-8a7b-7238d8e10b86 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:54.771562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-b4763bd9-872f-471a-8a7b-7238d8e10b86 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:54.778631 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b4763bd9-872f-471a-8a7b-7238d8e10b86 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:54.779163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b4763bd9-872f-471a-8a7b-7238d8e10b86 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:54.786123 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b4763bd9-872f-471a-8a7b-7238d8e10b86 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:54.786123 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b4763bd9-872f-471a-8a7b-7238d8e10b86 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:54.786123 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b4763bd9-872f-471a-8a7b-7238d8e10b86 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:54.786123 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-b4763bd9-872f-471a-8a7b-7238d8e10b86 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:54.794725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8274831d-7f62-474c-a28f-1be40c184fe7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:54.795335 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8274831d-7f62-474c-a28f-1be40c184fe7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:54.801378 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8274831d-7f62-474c-a28f-1be40c184fe7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:54.801867 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8274831d-7f62-474c-a28f-1be40c184fe7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:54.802431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8274831d-7f62-474c-a28f-1be40c184fe7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:54.802981 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8274831d-7f62-474c-a28f-1be40c184fe7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:54.803431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8274831d-7f62-474c-a28f-1be40c184fe7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:54.803859 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8274831d-7f62-474c-a28f-1be40c184fe7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:54.875860 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8274831d-7f62-474c-a28f-1be40c184fe7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:34:54.884248 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8274831d-7f62-474c-a28f-1be40c184fe7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1135 microversion: 2.1 time: 0.094001 Aug 21 01:34:54.885291 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 507/998] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:54 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1135 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:34:54.891387 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-b4763bd9-872f-471a-8a7b-7238d8e10b86 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lazy-loading 'services' on Instance uuid 8554ae19-faa1-4d72-8d84-529daa809281 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:34:55.157947 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-b4763bd9-872f-471a-8a7b-7238d8e10b86 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments/7d846d2e-b601-446e-9d5e-412a948137bc" status: 202 len: 0 microversion: 2.1 time: 0.390593 Aug 21 01:34:55.158592 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 492/999] 10.222.0.47 () {58 vars in 1416 bytes} [Sun Aug 21 01:34:54 2022] DELETE /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments/7d846d2e-b601-446e-9d5e-412a948137bc => generated 0 bytes in 392 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:34:55.167033 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4fdb68be-efab-4196-b45a-a87cff79c1eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:55.169307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4fdb68be-efab-4196-b45a-a87cff79c1eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:55.175230 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4fdb68be-efab-4196-b45a-a87cff79c1eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:55.175704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4fdb68be-efab-4196-b45a-a87cff79c1eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:55.176163 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4fdb68be-efab-4196-b45a-a87cff79c1eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:55.176847 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4fdb68be-efab-4196-b45a-a87cff79c1eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:55.177369 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4fdb68be-efab-4196-b45a-a87cff79c1eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:55.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4fdb68be-efab-4196-b45a-a87cff79c1eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:55.218097 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4fdb68be-efab-4196-b45a-a87cff79c1eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054907 Aug 21 01:34:55.218660 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 508/1000] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:34:55 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:34:55.908204 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3290c8cf-f79e-4322-9195-4ed03180cbc2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:55.908933 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3290c8cf-f79e-4322-9195-4ed03180cbc2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:55.915737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3290c8cf-f79e-4322-9195-4ed03180cbc2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:55.916975 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3290c8cf-f79e-4322-9195-4ed03180cbc2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:55.917431 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3290c8cf-f79e-4322-9195-4ed03180cbc2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:55.918147 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3290c8cf-f79e-4322-9195-4ed03180cbc2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:55.918651 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3290c8cf-f79e-4322-9195-4ed03180cbc2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:55.919096 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3290c8cf-f79e-4322-9195-4ed03180cbc2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:56.231504 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-35e67520-8fda-412e-b97e-5552e44d7390 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:56.234371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-35e67520-8fda-412e-b97e-5552e44d7390 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:56.241331 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-35e67520-8fda-412e-b97e-5552e44d7390 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:56.241821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-35e67520-8fda-412e-b97e-5552e44d7390 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:56.242447 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-35e67520-8fda-412e-b97e-5552e44d7390 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:56.243381 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-35e67520-8fda-412e-b97e-5552e44d7390 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:56.243983 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-35e67520-8fda-412e-b97e-5552e44d7390 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:56.244342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-35e67520-8fda-412e-b97e-5552e44d7390 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:56.277657 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-35e67520-8fda-412e-b97e-5552e44d7390 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050745 Aug 21 01:34:56.282748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 509/1001] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:34:56 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:34:56.299066 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3290c8cf-f79e-4322-9195-4ed03180cbc2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:34:56.307388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3290c8cf-f79e-4322-9195-4ed03180cbc2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1199 microversion: 2.1 time: 0.404999 Aug 21 01:34:56.307974 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 493/1002] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:55 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1199 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:34:57.290145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-14f04503-fd89-4835-8e1c-90fbf23de200 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:57.292873 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-14f04503-fd89-4835-8e1c-90fbf23de200 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:57.299695 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-14f04503-fd89-4835-8e1c-90fbf23de200 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:57.299943 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-14f04503-fd89-4835-8e1c-90fbf23de200 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:57.300418 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-14f04503-fd89-4835-8e1c-90fbf23de200 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:57.301124 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-14f04503-fd89-4835-8e1c-90fbf23de200 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:57.301580 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-14f04503-fd89-4835-8e1c-90fbf23de200 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:57.302005 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-14f04503-fd89-4835-8e1c-90fbf23de200 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:57.328449 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bde38090-9706-4c61-9000-000efaaf405e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:57.334459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-14f04503-fd89-4835-8e1c-90fbf23de200 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048429 Aug 21 01:34:57.335056 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 510/1003] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:34:57 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 21 01:34:57.337558 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bde38090-9706-4c61-9000-000efaaf405e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:57.343865 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bde38090-9706-4c61-9000-000efaaf405e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:57.344464 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bde38090-9706-4c61-9000-000efaaf405e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:57.344906 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bde38090-9706-4c61-9000-000efaaf405e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:57.345903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bde38090-9706-4c61-9000-000efaaf405e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:57.346542 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bde38090-9706-4c61-9000-000efaaf405e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:57.347146 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bde38090-9706-4c61-9000-000efaaf405e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:57.413321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-44c5949f-0b64-4623-b224-6030dc91e6a4 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:57.414901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-44c5949f-0b64-4623-b224-6030dc91e6a4 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:57.421756 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-44c5949f-0b64-4623-b224-6030dc91e6a4 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:57.421756 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-44c5949f-0b64-4623-b224-6030dc91e6a4 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:57.422899 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-44c5949f-0b64-4623-b224-6030dc91e6a4 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:57.423707 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-44c5949f-0b64-4623-b224-6030dc91e6a4 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:57.424347 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-44c5949f-0b64-4623-b224-6030dc91e6a4 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:57.424948 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-44c5949f-0b64-4623-b224-6030dc91e6a4 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:57.455761 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-44c5949f-0b64-4623-b224-6030dc91e6a4 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Going to try to terminate instance {{(pid=121277) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:34:57.580631 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-bde38090-9706-4c61-9000-000efaaf405e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:34:57.589355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bde38090-9706-4c61-9000-000efaaf405e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1277 microversion: 2.1 time: 0.264788 Aug 21 01:34:57.590290 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 494/1004] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:57 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1277 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:34:57.839941 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-44c5949f-0b64-4623-b224-6030dc91e6a4 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 204 len: 0 microversion: 2.1 time: 0.430326 Aug 21 01:34:57.840703 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 511/1005] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:34:57 2022] DELETE /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 21 01:34:57.849525 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e079decb-e90a-46da-b7a7-740803027936 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:57.850164 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e079decb-e90a-46da-b7a7-740803027936 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:57.856751 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e079decb-e90a-46da-b7a7-740803027936 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:57.857244 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e079decb-e90a-46da-b7a7-740803027936 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:57.857677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e079decb-e90a-46da-b7a7-740803027936 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:57.858374 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e079decb-e90a-46da-b7a7-740803027936 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:57.858818 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e079decb-e90a-46da-b7a7-740803027936 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:57.859239 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e079decb-e90a-46da-b7a7-740803027936 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:58.076517 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e079decb-e90a-46da-b7a7-740803027936 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:34:58.084556 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e079decb-e90a-46da-b7a7-740803027936 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1679 microversion: 2.1 time: 0.239172 Aug 21 01:34:58.085214 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 495/1006] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:57 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1679 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:34:58.250280 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-4d906dfb-9671-4096-ab7b-55bcbf6e8744 req-c5a8fd8d-0224-4a2e-9a2c-ea55a384f811 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "85c51937-c541-455e-a0db-4d06041c67be", "tag": "b8bc8818-25c9-4200-8011-9f19e85c6eec"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:34:58.260291 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-4d906dfb-9671-4096-ab7b-55bcbf6e8744 req-c5a8fd8d-0224-4a2e-9a2c-ea55a384f811 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:58.260834 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-4d906dfb-9671-4096-ab7b-55bcbf6e8744 req-c5a8fd8d-0224-4a2e-9a2c-ea55a384f811 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:58.261261 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-4d906dfb-9671-4096-ab7b-55bcbf6e8744 req-c5a8fd8d-0224-4a2e-9a2c-ea55a384f811 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:58.272713 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-4d906dfb-9671-4096-ab7b-55bcbf6e8744 req-c5a8fd8d-0224-4a2e-9a2c-ea55a384f811 service nova] Creating event network-changed:b8bc8818-25c9-4200-8011-9f19e85c6eec for instance 85c51937-c541-455e-a0db-4d06041c67be on hv-cinder-maste Aug 21 01:34:58.277740 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-4d906dfb-9671-4096-ab7b-55bcbf6e8744 req-c5a8fd8d-0224-4a2e-9a2c-ea55a384f811 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031521 Aug 21 01:34:58.284369 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 512/1007] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:34:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:34:58.612138 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9b5dcde8-25cb-43a6-a868-ac4ba249c734 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:58.612708 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9b5dcde8-25cb-43a6-a868-ac4ba249c734 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:58.620039 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9b5dcde8-25cb-43a6-a868-ac4ba249c734 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:58.620575 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9b5dcde8-25cb-43a6-a868-ac4ba249c734 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:58.621026 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9b5dcde8-25cb-43a6-a868-ac4ba249c734 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:58.621742 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9b5dcde8-25cb-43a6-a868-ac4ba249c734 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:58.622199 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9b5dcde8-25cb-43a6-a868-ac4ba249c734 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:58.622633 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9b5dcde8-25cb-43a6-a868-ac4ba249c734 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:58.764283 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9b5dcde8-25cb-43a6-a868-ac4ba249c734 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:34:58.772031 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9b5dcde8-25cb-43a6-a868-ac4ba249c734 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1277 microversion: 2.1 time: 0.163864 Aug 21 01:34:58.772635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 496/1008] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:58 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1277 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:34:59.107923 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b7b213b2-2789-4c3f-bae7-a63ff04e437a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:59.108676 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b7b213b2-2789-4c3f-bae7-a63ff04e437a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:59.115362 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b7b213b2-2789-4c3f-bae7-a63ff04e437a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:59.115846 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b7b213b2-2789-4c3f-bae7-a63ff04e437a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:59.116325 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b7b213b2-2789-4c3f-bae7-a63ff04e437a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:59.117017 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b7b213b2-2789-4c3f-bae7-a63ff04e437a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:59.117462 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b7b213b2-2789-4c3f-bae7-a63ff04e437a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:59.117940 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b7b213b2-2789-4c3f-bae7-a63ff04e437a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:59.248265 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b7b213b2-2789-4c3f-bae7-a63ff04e437a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:34:59.256145 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b7b213b2-2789-4c3f-bae7-a63ff04e437a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1679 microversion: 2.1 time: 0.152976 Aug 21 01:34:59.256719 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 513/1009] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:59 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1679 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:34:59.792675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-78a30abe-8f58-4c65-ab42-60eb290f6d09 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:34:59.793288 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-78a30abe-8f58-4c65-ab42-60eb290f6d09 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:34:59.799843 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-78a30abe-8f58-4c65-ab42-60eb290f6d09 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:59.800366 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-78a30abe-8f58-4c65-ab42-60eb290f6d09 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:59.800822 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-78a30abe-8f58-4c65-ab42-60eb290f6d09 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:34:59.801519 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-78a30abe-8f58-4c65-ab42-60eb290f6d09 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:34:59.801972 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-78a30abe-8f58-4c65-ab42-60eb290f6d09 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:34:59.802391 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-78a30abe-8f58-4c65-ab42-60eb290f6d09 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:00.065292 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-78a30abe-8f58-4c65-ab42-60eb290f6d09 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:00.076693 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-78a30abe-8f58-4c65-ab42-60eb290f6d09 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1277 microversion: 2.1 time: 0.287679 Aug 21 01:35:00.077717 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 497/1010] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:34:59 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1277 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:00.277280 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a161225e-5d8c-4c7f-991b-08901eca1c56 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:00.277901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a161225e-5d8c-4c7f-991b-08901eca1c56 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:00.285141 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a161225e-5d8c-4c7f-991b-08901eca1c56 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:00.285638 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a161225e-5d8c-4c7f-991b-08901eca1c56 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:00.286079 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a161225e-5d8c-4c7f-991b-08901eca1c56 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:00.286783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a161225e-5d8c-4c7f-991b-08901eca1c56 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:00.287241 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a161225e-5d8c-4c7f-991b-08901eca1c56 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:00.287666 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a161225e-5d8c-4c7f-991b-08901eca1c56 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:00.622198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a161225e-5d8c-4c7f-991b-08901eca1c56 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:00.630407 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a161225e-5d8c-4c7f-991b-08901eca1c56 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1679 microversion: 2.1 time: 0.357063 Aug 21 01:35:00.631002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 514/1011] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:00 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1679 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:01.099167 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-12f29530-e43c-42c4-a9d7-73bf3dd58b5c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:01.099857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-12f29530-e43c-42c4-a9d7-73bf3dd58b5c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:01.106910 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-12f29530-e43c-42c4-a9d7-73bf3dd58b5c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:01.107400 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-12f29530-e43c-42c4-a9d7-73bf3dd58b5c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:01.107939 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-12f29530-e43c-42c4-a9d7-73bf3dd58b5c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:01.108673 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-12f29530-e43c-42c4-a9d7-73bf3dd58b5c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:01.109150 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-12f29530-e43c-42c4-a9d7-73bf3dd58b5c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:01.109574 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-12f29530-e43c-42c4-a9d7-73bf3dd58b5c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:01.496612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-2b26a92e-5444-47dc-8ab0-1e4886745b51 req-fd51cf97-2b7d-4c51-854f-caf2069a81fe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8554ae19-faa1-4d72-8d84-529daa809281", "name": "network-vif-deleted", "tag": "7d30bbc9-a97b-43d2-b319-9d4924803d0f"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:35:01.504685 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-12f29530-e43c-42c4-a9d7-73bf3dd58b5c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:01.509345 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-12f29530-e43c-42c4-a9d7-73bf3dd58b5c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1265 microversion: 2.1 time: 0.413907 Aug 21 01:35:01.510976 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 498/1012] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:01 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1265 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:01.513344 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-2b26a92e-5444-47dc-8ab0-1e4886745b51 req-fd51cf97-2b7d-4c51-854f-caf2069a81fe service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:01.514110 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-2b26a92e-5444-47dc-8ab0-1e4886745b51 req-fd51cf97-2b7d-4c51-854f-caf2069a81fe service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:01.514741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-2b26a92e-5444-47dc-8ab0-1e4886745b51 req-fd51cf97-2b7d-4c51-854f-caf2069a81fe service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:01.527875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-2b26a92e-5444-47dc-8ab0-1e4886745b51 req-fd51cf97-2b7d-4c51-854f-caf2069a81fe service nova] Creating event network-vif-deleted:7d30bbc9-a97b-43d2-b319-9d4924803d0f for instance 8554ae19-faa1-4d72-8d84-529daa809281 on hv-cinder-maste Aug 21 01:35:01.533260 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-2b26a92e-5444-47dc-8ab0-1e4886745b51 req-fd51cf97-2b7d-4c51-854f-caf2069a81fe service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045860 Aug 21 01:35:01.534271 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 515/1013] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:35:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:35:01.651617 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c8e10045-fca1-420c-98a6-74990ae90823 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:01.652858 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c8e10045-fca1-420c-98a6-74990ae90823 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:01.659681 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8e10045-fca1-420c-98a6-74990ae90823 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:01.660493 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8e10045-fca1-420c-98a6-74990ae90823 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:01.661041 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8e10045-fca1-420c-98a6-74990ae90823 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:01.661860 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8e10045-fca1-420c-98a6-74990ae90823 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:01.662419 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8e10045-fca1-420c-98a6-74990ae90823 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:01.663158 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8e10045-fca1-420c-98a6-74990ae90823 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:01.733327 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c8e10045-fca1-420c-98a6-74990ae90823 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:01.741123 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c8e10045-fca1-420c-98a6-74990ae90823 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 200 len: 1611 microversion: 2.1 time: 0.093836 Aug 21 01:35:01.745055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 499/1014] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:01 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 1611 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:02.532045 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b177cce7-0ab3-45e7-a356-9f375787dfbe tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:02.532976 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b177cce7-0ab3-45e7-a356-9f375787dfbe tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:02.539885 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b177cce7-0ab3-45e7-a356-9f375787dfbe tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:02.540487 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b177cce7-0ab3-45e7-a356-9f375787dfbe tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:02.541179 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b177cce7-0ab3-45e7-a356-9f375787dfbe tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:02.541945 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b177cce7-0ab3-45e7-a356-9f375787dfbe tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:02.542408 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b177cce7-0ab3-45e7-a356-9f375787dfbe tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:02.542836 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b177cce7-0ab3-45e7-a356-9f375787dfbe tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:02.675040 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:02.763386 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b177cce7-0ab3-45e7-a356-9f375787dfbe tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:02.765683 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-cb50bddc-c5f5-4220-bd8a-00395bff6bad tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:02.766300 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-cb50bddc-c5f5-4220-bd8a-00395bff6bad tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:02.768304 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:02.771710 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b177cce7-0ab3-45e7-a356-9f375787dfbe tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1265 microversion: 2.1 time: 0.243849 Aug 21 01:35:02.772299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 516/1015] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:02 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1265 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:35:02.779251 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cb50bddc-c5f5-4220-bd8a-00395bff6bad tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:02.780966 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cb50bddc-c5f5-4220-bd8a-00395bff6bad tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:02.780966 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cb50bddc-c5f5-4220-bd8a-00395bff6bad tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:02.781228 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cb50bddc-c5f5-4220-bd8a-00395bff6bad tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:02.781703 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cb50bddc-c5f5-4220-bd8a-00395bff6bad tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:02.782060 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cb50bddc-c5f5-4220-bd8a-00395bff6bad tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:02.796695 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-cb50bddc-c5f5-4220-bd8a-00395bff6bad tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-cb50bddc-c5f5-4220-bd8a-00395bff6bad tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 HTTP/1.1 Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:35:02.798389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:35:02.800980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:35:02.800980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:35:02.800980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:35:02.800980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:35:02.800980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:35:02.800980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:35:02.800980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:35:02.800980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:35:02.800980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:35:02.800980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:35:02.800980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:35:02.800980 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '8554ae19-faa1-4d72-8d84-529daa809281'} Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:35:02.802425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:35:02.804999 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:35:02.807963 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-cb50bddc-c5f5-4220-bd8a-00395bff6bad tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 404 to user: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:35:02.807963 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-cb50bddc-c5f5-4220-bd8a-00395bff6bad tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 404 len: 111 microversion: 2.1 time: 0.037979 Aug 21 01:35:02.807963 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 500/1016] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:02 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 21 01:35:02.810903 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c4f87369-cf11-4d7a-b971-630b97393c3d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '.version_select at 0x7f9da34ead30>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:02.937000 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c4f87369-cf11-4d7a-b971-630b97393c3d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1633583235" status: 202 len: 0 microversion: 2.1 time: 0.131824 Aug 21 01:35:02.937503 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 517/1017] 10.222.0.47 () {58 vars in 1221 bytes} [Sun Aug 21 01:35:02 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1633583235 => generated 0 bytes in 133 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:35:03.729611 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1b0edbd1-4691-42e3-82ca-b031d075d31b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: .version_select at 0x7f9da32f9ee0>, body: {"keypair": {"name": "tempest-keypair-1058068127", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKs9Fca2icz70aeX8r8Fxgqn5zbOWGumSr+LbyetbmKiAGhraqNVSgXYlOZRVrXvdLa6vMcr6HBPNwbsAoZivRg2ZCPqeru+GEYqv1d9zLuqNtXR9jWQV/1wROdqGfKkJg=="}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:35:03.761853 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-1b0edbd1-4691-42e3-82ca-b031d075d31b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for project 122b46d9e29843a88c7ca5f019426539. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:35:03.761853 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-1b0edbd1-4691-42e3-82ca-b031d075d31b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for user a91ac0ba7e8c4418930852b0fa184247 and project 122b46d9e29843a88c7ca5f019426539. Resources: {'key_pairs'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:35:03.801729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-683c4b64-278d-4ff2-81f4-7c0f59e42232 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:03.802466 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-683c4b64-278d-4ff2-81f4-7c0f59e42232 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:03.808620 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-683c4b64-278d-4ff2-81f4-7c0f59e42232 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:03.809114 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-683c4b64-278d-4ff2-81f4-7c0f59e42232 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:03.809550 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-683c4b64-278d-4ff2-81f4-7c0f59e42232 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:03.810265 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-683c4b64-278d-4ff2-81f4-7c0f59e42232 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:03.810724 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-683c4b64-278d-4ff2-81f4-7c0f59e42232 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:03.811149 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-683c4b64-278d-4ff2-81f4-7c0f59e42232 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:03.868743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1b0edbd1-4691-42e3-82ca-b031d075d31b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.144640 Aug 21 01:35:03.869403 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 501/1018] 10.222.0.47 () {60 vars in 1159 bytes} [Sun Aug 21 01:35:03 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:35:03.957753 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-683c4b64-278d-4ff2-81f4-7c0f59e42232 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:03.968358 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-683c4b64-278d-4ff2-81f4-7c0f59e42232 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1265 microversion: 2.1 time: 0.172712 Aug 21 01:35:03.968942 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 518/1019] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:03 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1265 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:35:04.989601 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-7bf61614-c7ae-4f31-a109-f34d39df0c54 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:04.990239 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-7bf61614-c7ae-4f31-a109-f34d39df0c54 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:04.997457 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7bf61614-c7ae-4f31-a109-f34d39df0c54 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:04.998793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7bf61614-c7ae-4f31-a109-f34d39df0c54 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:04.998793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7bf61614-c7ae-4f31-a109-f34d39df0c54 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:04.999902 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7bf61614-c7ae-4f31-a109-f34d39df0c54 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:04.999902 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7bf61614-c7ae-4f31-a109-f34d39df0c54 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:04.999902 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-7bf61614-c7ae-4f31-a109-f34d39df0c54 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:05.151946 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-7bf61614-c7ae-4f31-a109-f34d39df0c54 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:05.160782 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-7bf61614-c7ae-4f31-a109-f34d39df0c54 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1265 microversion: 2.1 time: 0.175516 Aug 21 01:35:05.161386 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 502/1020] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:04 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1265 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:35:06.181969 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-79d332db-9522-45ee-a76b-049207dcff7e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:06.182612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-79d332db-9522-45ee-a76b-049207dcff7e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:06.189210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79d332db-9522-45ee-a76b-049207dcff7e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:06.189701 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79d332db-9522-45ee-a76b-049207dcff7e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:06.190180 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79d332db-9522-45ee-a76b-049207dcff7e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:06.190881 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79d332db-9522-45ee-a76b-049207dcff7e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:06.191337 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79d332db-9522-45ee-a76b-049207dcff7e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:06.191761 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79d332db-9522-45ee-a76b-049207dcff7e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:06.327339 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-15585047", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "99061e98-94a6-4cdf-b3c1-c9af69a6346c"}], "security_groups": [{"name": "tempest-securitygroup--1126799713"}], "key_name": "tempest-keypair-1058068127", "user_data": "}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:35:06.354614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-79d332db-9522-45ee-a76b-049207dcff7e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:06.367098 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-79d332db-9522-45ee-a76b-049207dcff7e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1265 microversion: 2.1 time: 0.189366 Aug 21 01:35:06.368036 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 519/1021] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:06 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1265 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:06.443639 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] validate_networks() for [('99061e98-94a6-4cdf-b3c1-c9af69a6346c', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:35:06.843622 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:35:06.844043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:35:06.844566 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:35:06.844979 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:35:06.845400 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:35:06.855026 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:06.855511 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:06.855953 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:06.856718 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:06.857177 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:06.857605 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:06.874534 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for project 122b46d9e29843a88c7ca5f019426539. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:35:06.878511 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Getting quotas for user a91ac0ba7e8c4418930852b0fa184247 and project 122b46d9e29843a88c7ca5f019426539. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:35:06.893232 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:35:06.895447 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:35:07.068894 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-81eb2c43-e00f-4c62-a229-4fd6f364304d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.746385 Aug 21 01:35:07.069531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 503/1022] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:35:06 2022] POST /compute/v2.1/servers => generated 402 bytes in 748 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 21 01:35:07.081188 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f5dab2cf-d255-4bbb-8056-bc304d2aa5c2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:07.082016 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f5dab2cf-d255-4bbb-8056-bc304d2aa5c2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:07.141985 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f5dab2cf-d255-4bbb-8056-bc304d2aa5c2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:07.150898 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f5dab2cf-d255-4bbb-8056-bc304d2aa5c2 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1240 microversion: 2.1 time: 0.073054 Aug 21 01:35:07.151496 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 520/1023] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:07 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1240 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:07.389759 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ce89400d-afe8-49c0-ae8b-7513187f1acf tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:07.390371 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ce89400d-afe8-49c0-ae8b-7513187f1acf tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:07.397691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce89400d-afe8-49c0-ae8b-7513187f1acf tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:07.398173 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce89400d-afe8-49c0-ae8b-7513187f1acf tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:07.398566 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce89400d-afe8-49c0-ae8b-7513187f1acf tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:07.399303 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce89400d-afe8-49c0-ae8b-7513187f1acf tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:07.399758 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce89400d-afe8-49c0-ae8b-7513187f1acf tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:07.400215 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce89400d-afe8-49c0-ae8b-7513187f1acf tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:07.539898 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ce89400d-afe8-49c0-ae8b-7513187f1acf tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:07.548776 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ce89400d-afe8-49c0-ae8b-7513187f1acf tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1265 microversion: 2.1 time: 0.162830 Aug 21 01:35:07.549799 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 504/1024] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:07 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1265 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:35:08.173323 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-6ceef129-24b1-46a7-b926-e2aec54c367a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:08.174202 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-6ceef129-24b1-46a7-b926-e2aec54c367a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:08.181865 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6ceef129-24b1-46a7-b926-e2aec54c367a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:08.182663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6ceef129-24b1-46a7-b926-e2aec54c367a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:08.182858 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6ceef129-24b1-46a7-b926-e2aec54c367a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:08.183521 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6ceef129-24b1-46a7-b926-e2aec54c367a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:08.183978 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6ceef129-24b1-46a7-b926-e2aec54c367a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:08.184438 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-6ceef129-24b1-46a7-b926-e2aec54c367a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:08.249796 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-6ceef129-24b1-46a7-b926-e2aec54c367a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:08.253274 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:08.257857 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-6ceef129-24b1-46a7-b926-e2aec54c367a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1244 microversion: 2.1 time: 0.090092 Aug 21 01:35:08.258422 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 521/1025] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:08 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1244 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:08.570333 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e0b4b7d3-cca2-42e2-b10c-8393964fb6a9 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:08.571048 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e0b4b7d3-cca2-42e2-b10c-8393964fb6a9 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:08.578365 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e0b4b7d3-cca2-42e2-b10c-8393964fb6a9 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:08.578849 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e0b4b7d3-cca2-42e2-b10c-8393964fb6a9 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:08.579282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e0b4b7d3-cca2-42e2-b10c-8393964fb6a9 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:08.580038 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e0b4b7d3-cca2-42e2-b10c-8393964fb6a9 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:08.580473 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e0b4b7d3-cca2-42e2-b10c-8393964fb6a9 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:08.580892 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e0b4b7d3-cca2-42e2-b10c-8393964fb6a9 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:08.722351 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e0b4b7d3-cca2-42e2-b10c-8393964fb6a9 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:08.730167 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e0b4b7d3-cca2-42e2-b10c-8393964fb6a9 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1265 microversion: 2.1 time: 0.163920 Aug 21 01:35:08.730749 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 505/1026] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:08 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1265 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:09.278944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-eb594061-f49b-4e7d-879c-120b05e40322 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:09.279498 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-eb594061-f49b-4e7d-879c-120b05e40322 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:09.286244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-eb594061-f49b-4e7d-879c-120b05e40322 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:09.286733 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-eb594061-f49b-4e7d-879c-120b05e40322 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:09.287175 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-eb594061-f49b-4e7d-879c-120b05e40322 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:09.287876 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-eb594061-f49b-4e7d-879c-120b05e40322 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:09.288369 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-eb594061-f49b-4e7d-879c-120b05e40322 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:09.288800 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-eb594061-f49b-4e7d-879c-120b05e40322 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:09.352887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-eb594061-f49b-4e7d-879c-120b05e40322 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:09.361648 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-eb594061-f49b-4e7d-879c-120b05e40322 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1292 microversion: 2.1 time: 0.087099 Aug 21 01:35:09.362225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 522/1027] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:09 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1292 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:09.750768 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-5747c6d3-1e02-450d-852b-ee89f27ddc3c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:09.751403 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-5747c6d3-1e02-450d-852b-ee89f27ddc3c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:09.758106 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5747c6d3-1e02-450d-852b-ee89f27ddc3c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:09.758603 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5747c6d3-1e02-450d-852b-ee89f27ddc3c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:09.759057 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5747c6d3-1e02-450d-852b-ee89f27ddc3c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:09.759802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5747c6d3-1e02-450d-852b-ee89f27ddc3c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:09.760304 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5747c6d3-1e02-450d-852b-ee89f27ddc3c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:09.760739 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5747c6d3-1e02-450d-852b-ee89f27ddc3c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:09.970586 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5747c6d3-1e02-450d-852b-ee89f27ddc3c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:09.978761 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5747c6d3-1e02-450d-852b-ee89f27ddc3c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1265 microversion: 2.1 time: 0.231844 Aug 21 01:35:09.981590 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 506/1028] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:09 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1265 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:10.383844 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d4268712-fcbb-408b-893c-5069058322e9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:10.385074 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d4268712-fcbb-408b-893c-5069058322e9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:10.392351 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d4268712-fcbb-408b-893c-5069058322e9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:10.392847 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d4268712-fcbb-408b-893c-5069058322e9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:10.393359 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d4268712-fcbb-408b-893c-5069058322e9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:10.394065 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d4268712-fcbb-408b-893c-5069058322e9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:10.394746 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d4268712-fcbb-408b-893c-5069058322e9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:10.395210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d4268712-fcbb-408b-893c-5069058322e9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:10.474000 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d4268712-fcbb-408b-893c-5069058322e9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:10.484945 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d4268712-fcbb-408b-893c-5069058322e9 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1298 microversion: 2.1 time: 0.105610 Aug 21 01:35:10.496542 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 523/1029] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:10 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1298 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:10.578120 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-a612936c-12e5-4e11-981a-ce37e4c9cbbd req-0ed28c53-895b-4271-a00f-59e5538f4f4b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "85c51937-c541-455e-a0db-4d06041c67be", "name": "network-vif-plugged", "status": "completed", "tag": "b8bc8818-25c9-4200-8011-9f19e85c6eec"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:35:10.588129 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-a612936c-12e5-4e11-981a-ce37e4c9cbbd req-0ed28c53-895b-4271-a00f-59e5538f4f4b service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:10.588662 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-a612936c-12e5-4e11-981a-ce37e4c9cbbd req-0ed28c53-895b-4271-a00f-59e5538f4f4b service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:10.590875 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-a612936c-12e5-4e11-981a-ce37e4c9cbbd req-0ed28c53-895b-4271-a00f-59e5538f4f4b service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:10.601049 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-a612936c-12e5-4e11-981a-ce37e4c9cbbd req-0ed28c53-895b-4271-a00f-59e5538f4f4b service nova] Creating event network-vif-plugged:b8bc8818-25c9-4200-8011-9f19e85c6eec for instance 85c51937-c541-455e-a0db-4d06041c67be on hv-cinder-maste Aug 21 01:35:10.606758 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-a612936c-12e5-4e11-981a-ce37e4c9cbbd req-0ed28c53-895b-4271-a00f-59e5538f4f4b service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034714 Aug 21 01:35:10.607592 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 507/1030] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:35:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:35:11.003143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b97c9127-46d1-4612-9230-66d7ca5a6524 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:11.003852 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b97c9127-46d1-4612-9230-66d7ca5a6524 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:11.010852 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b97c9127-46d1-4612-9230-66d7ca5a6524 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:11.011400 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b97c9127-46d1-4612-9230-66d7ca5a6524 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:11.011853 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b97c9127-46d1-4612-9230-66d7ca5a6524 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:11.012621 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b97c9127-46d1-4612-9230-66d7ca5a6524 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:11.013081 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b97c9127-46d1-4612-9230-66d7ca5a6524 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:11.013535 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b97c9127-46d1-4612-9230-66d7ca5a6524 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:11.286679 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b97c9127-46d1-4612-9230-66d7ca5a6524 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:11.295477 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b97c9127-46d1-4612-9230-66d7ca5a6524 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1265 microversion: 2.1 time: 0.296479 Aug 21 01:35:11.296125 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 524/1031] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:10 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1265 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:11.510956 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-0d0f90cf-af34-43c4-b12a-6a04ce90df0d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:11.511567 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-0d0f90cf-af34-43c4-b12a-6a04ce90df0d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:11.517849 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0d0f90cf-af34-43c4-b12a-6a04ce90df0d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:11.518329 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0d0f90cf-af34-43c4-b12a-6a04ce90df0d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:11.518765 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0d0f90cf-af34-43c4-b12a-6a04ce90df0d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:11.519452 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0d0f90cf-af34-43c4-b12a-6a04ce90df0d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:11.519902 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0d0f90cf-af34-43c4-b12a-6a04ce90df0d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:11.520387 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-0d0f90cf-af34-43c4-b12a-6a04ce90df0d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:11.747258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-0d0f90cf-af34-43c4-b12a-6a04ce90df0d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:11.755211 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-0d0f90cf-af34-43c4-b12a-6a04ce90df0d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1366 microversion: 2.1 time: 0.249590 Aug 21 01:35:11.755912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 508/1032] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:11 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1366 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:11.980595 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-ada9752a-f5f2-4868-9ce2-0aa56aa9d390 req-06748b94-cfec-4ae1-9db4-d40938ae04a3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a42c5c99-2ccb-452e-b2aa-a0c520e5e746", "tag": "e80f35e0-ecfc-432b-83a2-daae538d4c4f"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:35:11.991248 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ada9752a-f5f2-4868-9ce2-0aa56aa9d390 req-06748b94-cfec-4ae1-9db4-d40938ae04a3 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:11.991744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ada9752a-f5f2-4868-9ce2-0aa56aa9d390 req-06748b94-cfec-4ae1-9db4-d40938ae04a3 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:11.992212 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ada9752a-f5f2-4868-9ce2-0aa56aa9d390 req-06748b94-cfec-4ae1-9db4-d40938ae04a3 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:12.006744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-ada9752a-f5f2-4868-9ce2-0aa56aa9d390 req-06748b94-cfec-4ae1-9db4-d40938ae04a3 service nova] Creating event network-changed:e80f35e0-ecfc-432b-83a2-daae538d4c4f for instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 on hv-cinder-maste Aug 21 01:35:12.012659 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-ada9752a-f5f2-4868-9ce2-0aa56aa9d390 req-06748b94-cfec-4ae1-9db4-d40938ae04a3 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035846 Aug 21 01:35:12.013270 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 525/1033] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:35:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:35:12.317869 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2e692359-a941-4e01-917f-b6e7650584cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:12.318487 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2e692359-a941-4e01-917f-b6e7650584cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:12.325308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2e692359-a941-4e01-917f-b6e7650584cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:12.325799 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2e692359-a941-4e01-917f-b6e7650584cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:12.326449 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2e692359-a941-4e01-917f-b6e7650584cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:12.327430 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2e692359-a941-4e01-917f-b6e7650584cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:12.328092 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2e692359-a941-4e01-917f-b6e7650584cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:12.328753 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2e692359-a941-4e01-917f-b6e7650584cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:12.572254 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2e692359-a941-4e01-917f-b6e7650584cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:12.578404 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:12.581221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2e692359-a941-4e01-917f-b6e7650584cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1265 microversion: 2.1 time: 0.267175 Aug 21 01:35:12.581804 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 509/1034] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:12 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1265 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:35:12.779225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-09338615-60cf-4e2d-8549-20e6c5aba3d0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:12.779934 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-09338615-60cf-4e2d-8549-20e6c5aba3d0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:12.786280 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-09338615-60cf-4e2d-8549-20e6c5aba3d0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:12.786772 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-09338615-60cf-4e2d-8549-20e6c5aba3d0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:12.787217 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-09338615-60cf-4e2d-8549-20e6c5aba3d0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:12.787919 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-09338615-60cf-4e2d-8549-20e6c5aba3d0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:12.788405 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-09338615-60cf-4e2d-8549-20e6c5aba3d0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:12.788874 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-09338615-60cf-4e2d-8549-20e6c5aba3d0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:12.976143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-09338615-60cf-4e2d-8549-20e6c5aba3d0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:12.984967 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-09338615-60cf-4e2d-8549-20e6c5aba3d0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1366 microversion: 2.1 time: 0.210911 Aug 21 01:35:12.988068 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 526/1035] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:12 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1366 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:13.606869 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c18d237b-2f3f-4c60-aceb-66fe6bf5fca5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:13.607435 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c18d237b-2f3f-4c60-aceb-66fe6bf5fca5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:13.610055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=83,nova_api:INSERT=7 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:13.616028 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c18d237b-2f3f-4c60-aceb-66fe6bf5fca5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:13.616028 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c18d237b-2f3f-4c60-aceb-66fe6bf5fca5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:13.618314 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c18d237b-2f3f-4c60-aceb-66fe6bf5fca5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:13.618314 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c18d237b-2f3f-4c60-aceb-66fe6bf5fca5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:13.618314 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c18d237b-2f3f-4c60-aceb-66fe6bf5fca5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:13.618314 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c18d237b-2f3f-4c60-aceb-66fe6bf5fca5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:13.770947 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-c18d237b-2f3f-4c60-aceb-66fe6bf5fca5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:13.779056 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c18d237b-2f3f-4c60-aceb-66fe6bf5fca5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1452 microversion: 2.1 time: 0.176194 Aug 21 01:35:13.779699 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 510/1036] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:13 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1452 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:13.799255 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4f243783-bada-4726-9eb9-33033f919b77 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:13.799875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4f243783-bada-4726-9eb9-33033f919b77 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:13.806671 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f243783-bada-4726-9eb9-33033f919b77 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:13.807683 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f243783-bada-4726-9eb9-33033f919b77 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:13.808143 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f243783-bada-4726-9eb9-33033f919b77 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:13.808847 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f243783-bada-4726-9eb9-33033f919b77 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:13.809311 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f243783-bada-4726-9eb9-33033f919b77 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:13.809866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f243783-bada-4726-9eb9-33033f919b77 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:13.959934 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4f243783-bada-4726-9eb9-33033f919b77 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:13.968882 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4f243783-bada-4726-9eb9-33033f919b77 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1452 microversion: 2.1 time: 0.173532 Aug 21 01:35:13.969454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 527/1037] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:13 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1452 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:14.006107 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a806a9ce-118c-4779-9957-215e582403eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:14.006731 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a806a9ce-118c-4779-9957-215e582403eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:14.013012 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a806a9ce-118c-4779-9957-215e582403eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:14.013784 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a806a9ce-118c-4779-9957-215e582403eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:14.013928 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a806a9ce-118c-4779-9957-215e582403eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:14.014634 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a806a9ce-118c-4779-9957-215e582403eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:14.015090 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a806a9ce-118c-4779-9957-215e582403eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:14.015523 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a806a9ce-118c-4779-9957-215e582403eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:14.150088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a806a9ce-118c-4779-9957-215e582403eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:14.158323 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a806a9ce-118c-4779-9957-215e582403eb tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1366 microversion: 2.1 time: 0.156295 Aug 21 01:35:14.158875 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 511/1038] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:14 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1366 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:15.178965 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-aa94e50a-ca43-44ad-8f8e-445c3ef224c0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:15.179782 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-aa94e50a-ca43-44ad-8f8e-445c3ef224c0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:15.186881 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aa94e50a-ca43-44ad-8f8e-445c3ef224c0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:15.193515 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aa94e50a-ca43-44ad-8f8e-445c3ef224c0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:15.193515 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aa94e50a-ca43-44ad-8f8e-445c3ef224c0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:15.193515 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aa94e50a-ca43-44ad-8f8e-445c3ef224c0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:15.193515 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aa94e50a-ca43-44ad-8f8e-445c3ef224c0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:15.193879 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-aa94e50a-ca43-44ad-8f8e-445c3ef224c0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:15.326285 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-aa94e50a-ca43-44ad-8f8e-445c3ef224c0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:15.334213 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-aa94e50a-ca43-44ad-8f8e-445c3ef224c0 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1366 microversion: 2.1 time: 0.159032 Aug 21 01:35:15.334781 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 528/1039] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:15 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1366 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:16.185625 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-0f3ca8e4-0832-402a-a2a0-cfa000da60d4 req-e6815349-610f-4840-8221-f53dc4a4ae02 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "85c51937-c541-455e-a0db-4d06041c67be", "tag": "b8bc8818-25c9-4200-8011-9f19e85c6eec"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:35:16.196962 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-0f3ca8e4-0832-402a-a2a0-cfa000da60d4 req-e6815349-610f-4840-8221-f53dc4a4ae02 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:16.197531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-0f3ca8e4-0832-402a-a2a0-cfa000da60d4 req-e6815349-610f-4840-8221-f53dc4a4ae02 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:16.198859 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-0f3ca8e4-0832-402a-a2a0-cfa000da60d4 req-e6815349-610f-4840-8221-f53dc4a4ae02 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:16.211251 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-0f3ca8e4-0832-402a-a2a0-cfa000da60d4 req-e6815349-610f-4840-8221-f53dc4a4ae02 service nova] Creating event network-changed:b8bc8818-25c9-4200-8011-9f19e85c6eec for instance 85c51937-c541-455e-a0db-4d06041c67be on hv-cinder-maste Aug 21 01:35:16.217020 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-0f3ca8e4-0832-402a-a2a0-cfa000da60d4 req-e6815349-610f-4840-8221-f53dc4a4ae02 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035638 Aug 21 01:35:16.218211 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 512/1040] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:35:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:35:16.357791 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8d55803e-5496-4799-a40a-bcb556104259 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:16.359241 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8d55803e-5496-4799-a40a-bcb556104259 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:16.362439 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=61,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:16.368832 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d55803e-5496-4799-a40a-bcb556104259 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:16.370298 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d55803e-5496-4799-a40a-bcb556104259 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:16.370743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d55803e-5496-4799-a40a-bcb556104259 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:16.371654 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d55803e-5496-4799-a40a-bcb556104259 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:16.372634 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d55803e-5496-4799-a40a-bcb556104259 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:16.373060 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8d55803e-5496-4799-a40a-bcb556104259 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:16.592939 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8d55803e-5496-4799-a40a-bcb556104259 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:16.601737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8d55803e-5496-4799-a40a-bcb556104259 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1366 microversion: 2.1 time: 0.250661 Aug 21 01:35:16.602577 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 529/1041] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:16 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1366 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:16.778241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-b052985e-c1ce-4570-8119-39a03c41a1c3 req-15c1dd9d-2202-409f-b74d-7a5522ca96b0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a42c5c99-2ccb-452e-b2aa-a0c520e5e746", "name": "network-vif-plugged", "status": "completed", "tag": "e80f35e0-ecfc-432b-83a2-daae538d4c4f"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:35:16.787916 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-b052985e-c1ce-4570-8119-39a03c41a1c3 req-15c1dd9d-2202-409f-b74d-7a5522ca96b0 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:16.788425 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-b052985e-c1ce-4570-8119-39a03c41a1c3 req-15c1dd9d-2202-409f-b74d-7a5522ca96b0 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:16.788940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-b052985e-c1ce-4570-8119-39a03c41a1c3 req-15c1dd9d-2202-409f-b74d-7a5522ca96b0 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:16.800729 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-b052985e-c1ce-4570-8119-39a03c41a1c3 req-15c1dd9d-2202-409f-b74d-7a5522ca96b0 service nova] Creating event network-vif-plugged:e80f35e0-ecfc-432b-83a2-daae538d4c4f for instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 on hv-cinder-maste Aug 21 01:35:16.806287 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-b052985e-c1ce-4570-8119-39a03c41a1c3 req-15c1dd9d-2202-409f-b74d-7a5522ca96b0 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032775 Aug 21 01:35:16.806898 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 513/1042] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:35:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:35:17.624074 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3b4ca95f-a443-4532-b780-6673512868fd tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:17.624827 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3b4ca95f-a443-4532-b780-6673512868fd tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:17.628158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:17.633507 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3b4ca95f-a443-4532-b780-6673512868fd tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:17.634011 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3b4ca95f-a443-4532-b780-6673512868fd tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:17.634449 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3b4ca95f-a443-4532-b780-6673512868fd tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:17.635785 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3b4ca95f-a443-4532-b780-6673512868fd tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:17.636307 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3b4ca95f-a443-4532-b780-6673512868fd tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:17.636741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3b4ca95f-a443-4532-b780-6673512868fd tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:17.843876 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3b4ca95f-a443-4532-b780-6673512868fd tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:17.851732 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3b4ca95f-a443-4532-b780-6673512868fd tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1366 microversion: 2.1 time: 0.232664 Aug 21 01:35:17.852308 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 530/1043] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:17 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1366 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:35:18.873471 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-5d71ea9d-f89c-41ec-8321-fc18380d6f89 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:18.874091 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-5d71ea9d-f89c-41ec-8321-fc18380d6f89 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:18.875990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:18.876798 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=18 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:18.883631 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d71ea9d-f89c-41ec-8321-fc18380d6f89 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:18.883946 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d71ea9d-f89c-41ec-8321-fc18380d6f89 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:18.884427 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d71ea9d-f89c-41ec-8321-fc18380d6f89 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:18.885135 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d71ea9d-f89c-41ec-8321-fc18380d6f89 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:18.885625 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d71ea9d-f89c-41ec-8321-fc18380d6f89 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:18.886045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5d71ea9d-f89c-41ec-8321-fc18380d6f89 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:19.020390 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5d71ea9d-f89c-41ec-8321-fc18380d6f89 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:19.028901 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5d71ea9d-f89c-41ec-8321-fc18380d6f89 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1366 microversion: 2.1 time: 0.159785 Aug 21 01:35:19.029475 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 514/1044] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:18 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:20.050158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5da5ed3b-9206-4e39-bda3-7accb6bd8c76 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:20.050775 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5da5ed3b-9206-4e39-bda3-7accb6bd8c76 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:20.059018 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5da5ed3b-9206-4e39-bda3-7accb6bd8c76 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:20.059509 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5da5ed3b-9206-4e39-bda3-7accb6bd8c76 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:20.059941 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5da5ed3b-9206-4e39-bda3-7accb6bd8c76 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:20.060679 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5da5ed3b-9206-4e39-bda3-7accb6bd8c76 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:20.061134 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5da5ed3b-9206-4e39-bda3-7accb6bd8c76 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:20.061553 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5da5ed3b-9206-4e39-bda3-7accb6bd8c76 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:20.192134 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5da5ed3b-9206-4e39-bda3-7accb6bd8c76 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:35:20.199869 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5da5ed3b-9206-4e39-bda3-7accb6bd8c76 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1556 microversion: 2.1 time: 0.153595 Aug 21 01:35:20.200721 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 531/1045] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:35:20 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1556 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:35:20.220963 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bea8fd91-3830-44f9-a2f3-18d36e454342 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:20.221484 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bea8fd91-3830-44f9-a2f3-18d36e454342 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:20.227513 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bea8fd91-3830-44f9-a2f3-18d36e454342 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:20.227992 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bea8fd91-3830-44f9-a2f3-18d36e454342 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:20.228459 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bea8fd91-3830-44f9-a2f3-18d36e454342 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:20.229675 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bea8fd91-3830-44f9-a2f3-18d36e454342 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:20.230132 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bea8fd91-3830-44f9-a2f3-18d36e454342 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:20.230559 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bea8fd91-3830-44f9-a2f3-18d36e454342 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:20.310393 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bea8fd91-3830-44f9-a2f3-18d36e454342 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.093772 Aug 21 01:35:20.311012 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 515/1046] 10.222.0.47 () {58 vars in 1275 bytes} [Sun Aug 21 01:35:20 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/os-interface => generated 285 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:35:21.271328 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-ca51993e-f9a3-4d9e-86ef-b7a706406829 req-448818f1-7cea-4af9-9fb0-a004bb17d511 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a42c5c99-2ccb-452e-b2aa-a0c520e5e746", "tag": "e80f35e0-ecfc-432b-83a2-daae538d4c4f"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:35:21.281881 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ca51993e-f9a3-4d9e-86ef-b7a706406829 req-448818f1-7cea-4af9-9fb0-a004bb17d511 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:21.282453 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ca51993e-f9a3-4d9e-86ef-b7a706406829 req-448818f1-7cea-4af9-9fb0-a004bb17d511 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:21.283016 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ca51993e-f9a3-4d9e-86ef-b7a706406829 req-448818f1-7cea-4af9-9fb0-a004bb17d511 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:21.297664 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-ca51993e-f9a3-4d9e-86ef-b7a706406829 req-448818f1-7cea-4af9-9fb0-a004bb17d511 service nova] Creating event network-changed:e80f35e0-ecfc-432b-83a2-daae538d4c4f for instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 on hv-cinder-maste Aug 21 01:35:21.302825 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-ca51993e-f9a3-4d9e-86ef-b7a706406829 req-448818f1-7cea-4af9-9fb0-a004bb17d511 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035183 Aug 21 01:35:21.303487 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 532/1047] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:35:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:35:57.467494 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=11 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:57.468580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=26 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:57.474627 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-76b70346-0a34-470e-b420-549f48df7f5e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:35:57.475258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-76b70346-0a34-470e-b420-549f48df7f5e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:57.482086 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-76b70346-0a34-470e-b420-549f48df7f5e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:57.482594 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-76b70346-0a34-470e-b420-549f48df7f5e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:57.483036 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-76b70346-0a34-470e-b420-549f48df7f5e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:57.483748 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-76b70346-0a34-470e-b420-549f48df7f5e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:57.484235 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-76b70346-0a34-470e-b420-549f48df7f5e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:57.484662 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-76b70346-0a34-470e-b420-549f48df7f5e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:57.511026 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-76b70346-0a34-470e-b420-549f48df7f5e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/ips" status: 200 len: 148 microversion: 2.1 time: 0.045548 Aug 21 01:35:57.511607 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 516/1048] 10.222.0.47 () {58 vars in 1248 bytes} [Sun Aug 21 01:35:57 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/ips => generated 148 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:35:59.277850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=7 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:59.278621 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=58 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:35:59.282889 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3a6b5c87-a78d-4fce-9675-e4b434074b44 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b429a214-4251-4bf1-be2e-a20736cfc454"}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:35:59.285396 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3a6b5c87-a78d-4fce-9675-e4b434074b44 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:35:59.291400 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3a6b5c87-a78d-4fce-9675-e4b434074b44 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:59.291876 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3a6b5c87-a78d-4fce-9675-e4b434074b44 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:59.292338 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3a6b5c87-a78d-4fce-9675-e4b434074b44 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:35:59.293030 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3a6b5c87-a78d-4fce-9675-e4b434074b44 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:35:59.293573 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3a6b5c87-a78d-4fce-9675-e4b434074b44 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:35:59.293997 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3a6b5c87-a78d-4fce-9675-e4b434074b44 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:00.007203 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3a6b5c87-a78d-4fce-9675-e4b434074b44 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "POST /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.730946 Aug 21 01:36:00.007817 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 533/1049] 10.222.0.47 () {60 vars in 1323 bytes} [Sun Aug 21 01:35:59 2022] POST /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/os-volume_attachments => generated 194 bytes in 733 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:36:08.721143 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-76b4edb4-7da5-4733-94b5-2f1e3e858ed5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:08.721711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-76b4edb4-7da5-4733-94b5-2f1e3e858ed5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:08.723671 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:36:08.728672 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-76b4edb4-7da5-4733-94b5-2f1e3e858ed5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:08.729153 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-76b4edb4-7da5-4733-94b5-2f1e3e858ed5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:08.729588 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-76b4edb4-7da5-4733-94b5-2f1e3e858ed5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:08.730272 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-76b4edb4-7da5-4733-94b5-2f1e3e858ed5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:08.730764 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-76b4edb4-7da5-4733-94b5-2f1e3e858ed5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:08.731235 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-76b4edb4-7da5-4733-94b5-2f1e3e858ed5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:08.816293 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-76b4edb4-7da5-4733-94b5-2f1e3e858ed5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lazy-loading 'services' on Instance uuid a42c5c99-2ccb-452e-b2aa-a0c520e5e746 {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:36:09.133385 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-76b4edb4-7da5-4733-94b5-2f1e3e858ed5 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/os-volume_attachments/b429a214-4251-4bf1-be2e-a20736cfc454" status: 202 len: 0 microversion: 2.1 time: 0.416359 Aug 21 01:36:09.133974 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 517/1050] 10.222.0.47 () {58 vars in 1416 bytes} [Sun Aug 21 01:36:08 2022] DELETE /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/os-volume_attachments/b429a214-4251-4bf1-be2e-a20736cfc454 => generated 0 bytes in 418 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:36:09.143461 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-52b2038d-39e6-4b4f-a24f-c543b30787a3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:09.145939 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-52b2038d-39e6-4b4f-a24f-c543b30787a3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:09.152801 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-52b2038d-39e6-4b4f-a24f-c543b30787a3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:09.153308 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-52b2038d-39e6-4b4f-a24f-c543b30787a3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:09.153747 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-52b2038d-39e6-4b4f-a24f-c543b30787a3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:09.155042 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-52b2038d-39e6-4b4f-a24f-c543b30787a3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:09.155502 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-52b2038d-39e6-4b4f-a24f-c543b30787a3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:09.155932 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-52b2038d-39e6-4b4f-a24f-c543b30787a3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:09.186489 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-52b2038d-39e6-4b4f-a24f-c543b30787a3 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047812 Aug 21 01:36:09.187046 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 534/1051] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:36:09 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:36:10.200432 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-945bc7f7-9077-40d2-8fa4-1aceb9753253 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:10.221485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-945bc7f7-9077-40d2-8fa4-1aceb9753253 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:10.221485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-945bc7f7-9077-40d2-8fa4-1aceb9753253 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:10.221485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-945bc7f7-9077-40d2-8fa4-1aceb9753253 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:10.221485 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-945bc7f7-9077-40d2-8fa4-1aceb9753253 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:10.221931 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-945bc7f7-9077-40d2-8fa4-1aceb9753253 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:10.221931 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-945bc7f7-9077-40d2-8fa4-1aceb9753253 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:10.221931 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-945bc7f7-9077-40d2-8fa4-1aceb9753253 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:10.241535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-945bc7f7-9077-40d2-8fa4-1aceb9753253 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045945 Aug 21 01:36:10.242087 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 518/1052] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:36:10 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:36:11.254436 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-79c4c9cc-acb5-4f10-852c-2c7df981527b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:11.256631 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-79c4c9cc-acb5-4f10-852c-2c7df981527b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:11.263196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79c4c9cc-acb5-4f10-852c-2c7df981527b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:11.264422 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79c4c9cc-acb5-4f10-852c-2c7df981527b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:11.264918 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79c4c9cc-acb5-4f10-852c-2c7df981527b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:11.265742 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79c4c9cc-acb5-4f10-852c-2c7df981527b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:11.266261 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79c4c9cc-acb5-4f10-852c-2c7df981527b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:11.266728 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-79c4c9cc-acb5-4f10-852c-2c7df981527b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:11.297479 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-79c4c9cc-acb5-4f10-852c-2c7df981527b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047007 Aug 21 01:36:11.298060 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 535/1053] 10.222.0.47 () {58 vars in 1302 bytes} [Sun Aug 21 01:36:11 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Aug 21 01:36:11.361533 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-15f10a2e-8cbd-427f-81bf-fa5f2c889b9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:11.362179 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-15f10a2e-8cbd-427f-81bf-fa5f2c889b9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:11.368738 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-15f10a2e-8cbd-427f-81bf-fa5f2c889b9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:11.369175 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-15f10a2e-8cbd-427f-81bf-fa5f2c889b9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:11.369617 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-15f10a2e-8cbd-427f-81bf-fa5f2c889b9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:11.371975 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-15f10a2e-8cbd-427f-81bf-fa5f2c889b9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:11.371975 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-15f10a2e-8cbd-427f-81bf-fa5f2c889b9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:11.371975 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-15f10a2e-8cbd-427f-81bf-fa5f2c889b9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:11.398376 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-15f10a2e-8cbd-427f-81bf-fa5f2c889b9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:36:11.699909 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-15f10a2e-8cbd-427f-81bf-fa5f2c889b9b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 204 len: 0 microversion: 2.1 time: 0.341819 Aug 21 01:36:11.700499 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 519/1054] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:36:11 2022] DELETE /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:36:11.708901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f1d92d3e-c049-451c-a936-234bc48eeb2a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:11.711698 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f1d92d3e-c049-451c-a936-234bc48eeb2a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:11.715779 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f1d92d3e-c049-451c-a936-234bc48eeb2a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:11.716338 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f1d92d3e-c049-451c-a936-234bc48eeb2a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:11.716830 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f1d92d3e-c049-451c-a936-234bc48eeb2a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:11.717535 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f1d92d3e-c049-451c-a936-234bc48eeb2a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:11.718016 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f1d92d3e-c049-451c-a936-234bc48eeb2a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:11.718446 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f1d92d3e-c049-451c-a936-234bc48eeb2a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:11.848572 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f1d92d3e-c049-451c-a936-234bc48eeb2a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:36:11.865372 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f1d92d3e-c049-451c-a936-234bc48eeb2a tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1679 microversion: 2.1 time: 0.160318 Aug 21 01:36:11.865951 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 536/1055] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:36:11 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1679 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:36:12.889666 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8696543d-8937-4148-a8ef-c101731a3909 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:12.890371 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8696543d-8937-4148-a8ef-c101731a3909 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:12.897405 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8696543d-8937-4148-a8ef-c101731a3909 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:12.897889 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8696543d-8937-4148-a8ef-c101731a3909 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:12.898328 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8696543d-8937-4148-a8ef-c101731a3909 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:12.899022 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8696543d-8937-4148-a8ef-c101731a3909 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:12.899474 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8696543d-8937-4148-a8ef-c101731a3909 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:12.899898 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8696543d-8937-4148-a8ef-c101731a3909 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:13.044038 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8696543d-8937-4148-a8ef-c101731a3909 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:36:13.052019 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8696543d-8937-4148-a8ef-c101731a3909 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1679 microversion: 2.1 time: 0.167290 Aug 21 01:36:13.052618 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 520/1056] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:36:12 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1679 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:36:14.073612 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a11c101c-85a6-4344-a96c-df1e445a794e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:14.074175 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a11c101c-85a6-4344-a96c-df1e445a794e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:14.081807 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a11c101c-85a6-4344-a96c-df1e445a794e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:14.082356 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a11c101c-85a6-4344-a96c-df1e445a794e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:14.082745 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a11c101c-85a6-4344-a96c-df1e445a794e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:14.083446 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a11c101c-85a6-4344-a96c-df1e445a794e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:14.083905 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a11c101c-85a6-4344-a96c-df1e445a794e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:14.084613 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a11c101c-85a6-4344-a96c-df1e445a794e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:14.276003 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a11c101c-85a6-4344-a96c-df1e445a794e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:36:14.286809 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a11c101c-85a6-4344-a96c-df1e445a794e tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1679 microversion: 2.1 time: 0.217173 Aug 21 01:36:14.287473 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 537/1057] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:36:14 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1679 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:36:14.759489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-de6c59d8-a49c-41ba-b907-4da9dafa29c6 req-ca068282-e2f3-424f-a4f4-1aad57bfd5fb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a42c5c99-2ccb-452e-b2aa-a0c520e5e746", "name": "network-vif-deleted", "tag": "e80f35e0-ecfc-432b-83a2-daae538d4c4f"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:36:14.770751 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-de6c59d8-a49c-41ba-b907-4da9dafa29c6 req-ca068282-e2f3-424f-a4f4-1aad57bfd5fb service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:14.771467 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-de6c59d8-a49c-41ba-b907-4da9dafa29c6 req-ca068282-e2f3-424f-a4f4-1aad57bfd5fb service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:14.771968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-de6c59d8-a49c-41ba-b907-4da9dafa29c6 req-ca068282-e2f3-424f-a4f4-1aad57bfd5fb service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:14.784719 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-de6c59d8-a49c-41ba-b907-4da9dafa29c6 req-ca068282-e2f3-424f-a4f4-1aad57bfd5fb service nova] Creating event network-vif-deleted:e80f35e0-ecfc-432b-83a2-daae538d4c4f for instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 on hv-cinder-maste Aug 21 01:36:14.790045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-de6c59d8-a49c-41ba-b907-4da9dafa29c6 req-ca068282-e2f3-424f-a4f4-1aad57bfd5fb service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.098881 Aug 21 01:36:14.792526 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 521/1058] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:36:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:36:15.307149 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:15.307862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:15.314399 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:15.315019 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:15.315337 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:15.316040 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:15.316534 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:15.316968 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:15.353641 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:15.354129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:15.354569 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:15.355022 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lazy-loading 'fault' on Instance uuid a42c5c99-2ccb-452e-b2aa-a0c520e5e746 {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:36:15.397771 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a91ac0ba7e8c4418930852b0fa184247', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '122b46d9e29843a88c7ca5f019426539', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:36:15.406367 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-1fc6b84f-b501-4cd4-b048-b0acc032732f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 200 len: 1330 microversion: 2.1 time: 0.103189 Aug 21 01:36:15.406937 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 538/1059] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:36:15 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 1330 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:36:16.427916 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-03b23a36-5d90-42fb-9b1f-e0e76773097f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:16.428672 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-03b23a36-5d90-42fb-9b1f-e0e76773097f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:16.435382 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-03b23a36-5d90-42fb-9b1f-e0e76773097f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:16.435870 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-03b23a36-5d90-42fb-9b1f-e0e76773097f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:16.436930 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-03b23a36-5d90-42fb-9b1f-e0e76773097f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:16.437572 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-03b23a36-5d90-42fb-9b1f-e0e76773097f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:16.438022 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-03b23a36-5d90-42fb-9b1f-e0e76773097f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:16.438447 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-03b23a36-5d90-42fb-9b1f-e0e76773097f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:16.452134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-03b23a36-5d90-42fb-9b1f-e0e76773097f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-03b23a36-5d90-42fb-9b1f-e0e76773097f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 HTTP/1.1 Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:36:16.453831 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:36:16.455580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:36:16.455580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:16.455580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:16.455580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:16.455580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:16.455580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:36:16.455580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:36:16.455580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:36:16.455580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:16.455580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:36:16.455580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:16.455580 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': 'a42c5c99-2ccb-452e-b2aa-a0c520e5e746'} Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:36:16.456598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:16.457925 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:16.459112 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-03b23a36-5d90-42fb-9b1f-e0e76773097f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 404 to user: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:36:16.459112 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-03b23a36-5d90-42fb-9b1f-e0e76773097f tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 404 len: 111 microversion: 2.1 time: 0.031518 Aug 21 01:36:16.459112 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 522/1060] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:36:16 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:36:16.464423 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-65937406-1208-455f-8f5d-f7a9f7880620 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '.version_select at 0x7f9da31d2a60>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:16.546848 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-65937406-1208-455f-8f5d-f7a9f7880620 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1058068127" status: 202 len: 0 microversion: 2.1 time: 0.086275 Aug 21 01:36:16.547431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 539/1061] 10.222.0.47 () {58 vars in 1221 bytes} [Sun Aug 21 01:36:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1058068127 => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:36:18.568555 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bd90bcb5-98cc-4a54-954a-fbb46cf2ad2b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:18.569048 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bd90bcb5-98cc-4a54-954a-fbb46cf2ad2b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:18.575310 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bd90bcb5-98cc-4a54-954a-fbb46cf2ad2b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:18.575790 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bd90bcb5-98cc-4a54-954a-fbb46cf2ad2b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:18.576266 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bd90bcb5-98cc-4a54-954a-fbb46cf2ad2b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:18.577437 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bd90bcb5-98cc-4a54-954a-fbb46cf2ad2b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:18.577889 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bd90bcb5-98cc-4a54-954a-fbb46cf2ad2b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:18.578312 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bd90bcb5-98cc-4a54-954a-fbb46cf2ad2b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:18.594566 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-bd90bcb5-98cc-4a54-954a-fbb46cf2ad2b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bd90bcb5-98cc-4a54-954a-fbb46cf2ad2b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: DELETE /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 HTTP/1.1 Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:36:18.596088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:36:18.597755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:36:18.597755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:18.597755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:18.597755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:18.597755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:18.597755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:36:18.597755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:36:18.597755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:36:18.597755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:18.597755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:36:18.597755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:18.597755 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': 'a42c5c99-2ccb-452e-b2aa-a0c520e5e746'} Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:36:18.598642 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:18.599968 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:18.601694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bd90bcb5-98cc-4a54-954a-fbb46cf2ad2b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 404 to user: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:36:18.601694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bd90bcb5-98cc-4a54-954a-fbb46cf2ad2b tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 404 len: 111 microversion: 2.1 time: 0.032758 Aug 21 01:36:18.601694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 523/1062] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:36:18 2022] DELETE /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:36:18.606268 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b0cf283b-463f-4183-a98b-8ecba4eadc45 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:18.606887 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b0cf283b-463f-4183-a98b-8ecba4eadc45 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: a42c5c99-2ccb-452e-b2aa-a0c520e5e746] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:18.613575 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b0cf283b-463f-4183-a98b-8ecba4eadc45 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:18.614061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b0cf283b-463f-4183-a98b-8ecba4eadc45 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:18.614503 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b0cf283b-463f-4183-a98b-8ecba4eadc45 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:18.615198 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b0cf283b-463f-4183-a98b-8ecba4eadc45 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:18.615660 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b0cf283b-463f-4183-a98b-8ecba4eadc45 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:18.616149 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b0cf283b-463f-4183-a98b-8ecba4eadc45 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:18.630432 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-b0cf283b-463f-4183-a98b-8ecba4eadc45 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b0cf283b-463f-4183-a98b-8ecba4eadc45 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 HTTP/1.1 Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:36:18.632067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:36:18.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:36:18.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:18.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:18.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:18.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:18.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:36:18.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:36:18.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:36:18.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:18.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:36:18.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:18.633743 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': 'a42c5c99-2ccb-452e-b2aa-a0c520e5e746'} Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:36:18.634856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. Aug 21 01:36:18.636413 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:18.638067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b0cf283b-463f-4183-a98b-8ecba4eadc45 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 404 to user: Instance a42c5c99-2ccb-452e-b2aa-a0c520e5e746 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:36:18.638067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b0cf283b-463f-4183-a98b-8ecba4eadc45 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746" status: 404 len: 111 microversion: 2.1 time: 0.031527 Aug 21 01:36:18.638067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 540/1063] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:36:18 2022] GET /compute/v2.1/servers/a42c5c99-2ccb-452e-b2aa-a0c520e5e746 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:36:19.918276 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2aa45a59-8307-4c57-a0c0-9f7e1b575088 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:19.918794 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2aa45a59-8307-4c57-a0c0-9f7e1b575088 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:19.925732 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2aa45a59-8307-4c57-a0c0-9f7e1b575088 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:19.926213 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2aa45a59-8307-4c57-a0c0-9f7e1b575088 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:19.926646 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2aa45a59-8307-4c57-a0c0-9f7e1b575088 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:19.927347 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2aa45a59-8307-4c57-a0c0-9f7e1b575088 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:19.927796 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2aa45a59-8307-4c57-a0c0-9f7e1b575088 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:19.928260 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2aa45a59-8307-4c57-a0c0-9f7e1b575088 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:19.942505 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-2aa45a59-8307-4c57-a0c0-9f7e1b575088 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2aa45a59-8307-4c57-a0c0-9f7e1b575088 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: DELETE /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 HTTP/1.1 Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:36:19.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:36:19.945710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:36:19.945710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:19.945710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:19.945710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:19.945710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:19.945710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:36:19.945710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:36:19.945710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:36:19.945710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:19.945710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:36:19.945710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:36:19.945710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '8554ae19-faa1-4d72-8d84-529daa809281'} Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:36:19.946641 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:36:19.948258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:19.949837 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2aa45a59-8307-4c57-a0c0-9f7e1b575088 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 404 to user: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:36:19.949837 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2aa45a59-8307-4c57-a0c0-9f7e1b575088 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 404 len: 111 microversion: 2.1 time: 0.031015 Aug 21 01:36:19.949837 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 524/1064] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:36:19 2022] DELETE /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:36:19.954542 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b1e620ab-51bf-4d8f-9347-109534b28442 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:19.955181 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b1e620ab-51bf-4d8f-9347-109534b28442 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 8554ae19-faa1-4d72-8d84-529daa809281] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:19.961868 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b1e620ab-51bf-4d8f-9347-109534b28442 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:19.962342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b1e620ab-51bf-4d8f-9347-109534b28442 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:19.962771 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b1e620ab-51bf-4d8f-9347-109534b28442 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:19.963456 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b1e620ab-51bf-4d8f-9347-109534b28442 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:19.963902 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b1e620ab-51bf-4d8f-9347-109534b28442 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:19.964347 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b1e620ab-51bf-4d8f-9347-109534b28442 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:19.979835 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-b1e620ab-51bf-4d8f-9347-109534b28442 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b1e620ab-51bf-4d8f-9347-109534b28442 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 HTTP/1.1 Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:36:19.981428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:36:19.983158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:36:19.983158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:19.983158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:19.983158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:19.983158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:19.983158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:36:19.983158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:36:19.983158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:36:19.983158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:19.983158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:36:19.983158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:36:19.983158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '8554ae19-faa1-4d72-8d84-529daa809281'} Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:36:19.984194 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. Aug 21 01:36:19.985731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:19.986900 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b1e620ab-51bf-4d8f-9347-109534b28442 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 404 to user: Instance 8554ae19-faa1-4d72-8d84-529daa809281 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:36:19.986900 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b1e620ab-51bf-4d8f-9347-109534b28442 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281" status: 404 len: 111 microversion: 2.1 time: 0.032702 Aug 21 01:36:19.986900 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 541/1065] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:36:19 2022] GET /compute/v2.1/servers/8554ae19-faa1-4d72-8d84-529daa809281 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:36:19.991514 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bc26d4c7-29a8-4836-89f5-a5be6a3fa372 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:19.992016 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bc26d4c7-29a8-4836-89f5-a5be6a3fa372 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:19.998106 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc26d4c7-29a8-4836-89f5-a5be6a3fa372 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:19.998620 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc26d4c7-29a8-4836-89f5-a5be6a3fa372 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:19.999252 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc26d4c7-29a8-4836-89f5-a5be6a3fa372 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:19.999857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc26d4c7-29a8-4836-89f5-a5be6a3fa372 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:20.000362 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc26d4c7-29a8-4836-89f5-a5be6a3fa372 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:20.000792 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bc26d4c7-29a8-4836-89f5-a5be6a3fa372 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:20.015560 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-bc26d4c7-29a8-4836-89f5-a5be6a3fa372 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bc26d4c7-29a8-4836-89f5-a5be6a3fa372 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: DELETE /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 HTTP/1.1 Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:36:20.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:36:20.018827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:36:20.018827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:20.018827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:20.018827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:20.018827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:20.018827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:36:20.018827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:36:20.018827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:36:20.018827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:20.018827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:36:20.018827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:36:20.018827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '049eb468-fdda-4d9d-a362-b13443842254'} Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:36:20.019862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:36:20.021191 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:20.022568 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bc26d4c7-29a8-4836-89f5-a5be6a3fa372 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 404 to user: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:36:20.022568 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bc26d4c7-29a8-4836-89f5-a5be6a3fa372 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 404 len: 111 microversion: 2.1 time: 0.030928 Aug 21 01:36:20.022568 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 525/1066] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:36:19 2022] DELETE /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:36:20.027060 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4bc3b78a-d287-49b3-a178-2f35fef85e33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:20.027686 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4bc3b78a-d287-49b3-a178-2f35fef85e33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: 049eb468-fdda-4d9d-a362-b13443842254] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:20.034563 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4bc3b78a-d287-49b3-a178-2f35fef85e33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:20.035066 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4bc3b78a-d287-49b3-a178-2f35fef85e33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:20.035514 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4bc3b78a-d287-49b3-a178-2f35fef85e33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:20.036252 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4bc3b78a-d287-49b3-a178-2f35fef85e33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:20.036713 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4bc3b78a-d287-49b3-a178-2f35fef85e33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:20.037205 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4bc3b78a-d287-49b3-a178-2f35fef85e33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:20.051368 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-4bc3b78a-d287-49b3-a178-2f35fef85e33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4bc3b78a-d287-49b3-a178-2f35fef85e33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 HTTP/1.1 Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:36:20.052920 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:36:20.055454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:36:20.055454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:20.055454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:20.055454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:20.055454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:20.055454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:36:20.055454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:36:20.055454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:36:20.055454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:20.055454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:36:20.055454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:36:20.055454 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '049eb468-fdda-4d9d-a362-b13443842254'} Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:36:20.056614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. Aug 21 01:36:20.058704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:20.060821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4bc3b78a-d287-49b3-a178-2f35fef85e33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 404 to user: Instance 049eb468-fdda-4d9d-a362-b13443842254 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:36:20.060821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4bc3b78a-d287-49b3-a178-2f35fef85e33 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254" status: 404 len: 111 microversion: 2.1 time: 0.031090 Aug 21 01:36:20.060821 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 542/1067] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:36:20 2022] GET /compute/v2.1/servers/049eb468-fdda-4d9d-a362-b13443842254 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:36:21.328578 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2414c617-e07e-4730-9f45-e37af87b9b1d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:21.329135 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2414c617-e07e-4730-9f45-e37af87b9b1d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:21.335833 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2414c617-e07e-4730-9f45-e37af87b9b1d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:21.336358 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2414c617-e07e-4730-9f45-e37af87b9b1d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:21.336798 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2414c617-e07e-4730-9f45-e37af87b9b1d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:21.337508 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2414c617-e07e-4730-9f45-e37af87b9b1d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:21.337958 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2414c617-e07e-4730-9f45-e37af87b9b1d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:21.338397 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2414c617-e07e-4730-9f45-e37af87b9b1d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:21.353774 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-2414c617-e07e-4730-9f45-e37af87b9b1d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2414c617-e07e-4730-9f45-e37af87b9b1d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: DELETE /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 HTTP/1.1 Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:36:21.355338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:36:21.356993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:36:21.356993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:21.356993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:21.356993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:21.356993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:21.356993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:36:21.356993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:36:21.356993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:36:21.356993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:21.356993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:36:21.356993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:36:21.356993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': 'd1a9e859-eddf-4f45-8e02-168201b29a63'} Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:36:21.358055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:36:21.359598 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:36:21.361298 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2414c617-e07e-4730-9f45-e37af87b9b1d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 404 to user: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:36:21.361298 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2414c617-e07e-4730-9f45-e37af87b9b1d tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "DELETE /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 404 len: 111 microversion: 2.1 time: 0.032534 Aug 21 01:36:21.361298 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 526/1068] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:36:21 2022] DELETE /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:36:21.365459 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-305ef888-1dd8-4e5d-bcc4-13f01f131ba6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:36:21.366051 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-305ef888-1dd8-4e5d-bcc4-13f01f131ba6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] [instance: d1a9e859-eddf-4f45-8e02-168201b29a63] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:36:21.372942 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-305ef888-1dd8-4e5d-bcc4-13f01f131ba6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:21.373423 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-305ef888-1dd8-4e5d-bcc4-13f01f131ba6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:21.373856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-305ef888-1dd8-4e5d-bcc4-13f01f131ba6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:21.374556 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-305ef888-1dd8-4e5d-bcc4-13f01f131ba6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:36:21.375005 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-305ef888-1dd8-4e5d-bcc4-13f01f131ba6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:36:21.375426 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-305ef888-1dd8-4e5d-bcc4-13f01f131ba6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:36:21.391247 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-305ef888-1dd8-4e5d-bcc4-13f01f131ba6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] HTTP exception thrown: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-305ef888-1dd8-4e5d-bcc4-13f01f131ba6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Request method failure captured: Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 HTTP/1.1 Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYrdDSpUau-T8rC_0Xb9hdMkKDExgcRpKmDKrkEJxJ-IjKy1pjcNUFWfZE-TJIWb8yxzplLHgOglAefD5Jav9IIdwWp-xtgobNMn5fcog0y14VpIZWqJhuGOxAi6RiW5_dRt8VN2GaUrzBmWouwUmiSHf0wLPWiVzS2L2wvWPPUYgR0 Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:36:21.392855 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:36:21.394506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_122b46d9e29843a88c7ca5f019426539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/122b46d9e29843a88c7ca5f019426539"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:36:21.394506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:21.394506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 122b46d9e29843a88c7ca5f019426539 Aug 21 01:36:21.394506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-468680853 Aug 21 01:36:21.394506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:21.394506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:36:21.394506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:36:21.394506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: a91ac0ba7e8c4418930852b0fa184247 Aug 21 01:36:21.394506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-AttachVolumeNegativeTest-468680853-project Aug 21 01:36:21.394506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:36:21.394506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:36:21.394506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': 'd1a9e859-eddf-4f45-8e02-168201b29a63'} Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:36:21.395474 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. Aug 21 01:36:21.396862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:36:21.398049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-305ef888-1dd8-4e5d-bcc4-13f01f131ba6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] Returning 404 to user: Instance d1a9e859-eddf-4f45-8e02-168201b29a63 could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:36:21.398049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-305ef888-1dd8-4e5d-bcc4-13f01f131ba6 tempest-AttachVolumeNegativeTest-468680853 tempest-AttachVolumeNegativeTest-468680853-project] 10.222.0.47 "GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63" status: 404 len: 111 microversion: 2.1 time: 0.032756 Aug 21 01:36:21.398049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 543/1069] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:36:21 2022] GET /compute/v2.1/servers/d1a9e859-eddf-4f45-8e02-168201b29a63 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:36:55.963667 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:36:55.964410 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Sun Aug 21 01:36:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:37:13.213275 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-31e61f64-6387-4442-a90b-fedd5743f145 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:13.213866 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-31e61f64-6387-4442-a90b-fedd5743f145 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:13.216145 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=25,nova_api:UPDATE=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:37:13.216986 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=51,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:37:13.221556 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-31e61f64-6387-4442-a90b-fedd5743f145 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:13.222058 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-31e61f64-6387-4442-a90b-fedd5743f145 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:13.222489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-31e61f64-6387-4442-a90b-fedd5743f145 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:13.223184 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-31e61f64-6387-4442-a90b-fedd5743f145 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:13.224232 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-31e61f64-6387-4442-a90b-fedd5743f145 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:13.224667 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-31e61f64-6387-4442-a90b-fedd5743f145 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:13.253740 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-31e61f64-6387-4442-a90b-fedd5743f145 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:37:13.481037 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-31e61f64-6387-4442-a90b-fedd5743f145 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "DELETE /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 204 len: 0 microversion: 2.1 time: 0.271542 Aug 21 01:37:13.481698 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 527/1070] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:37:13 2022] DELETE /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 0 bytes in 274 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:37:13.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4e7565ce-0e48-49e6-be1b-8d8761560051 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:13.490616 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4e7565ce-0e48-49e6-be1b-8d8761560051 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:13.492850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=25,nova_api:DELETE=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:37:13.493491 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=51,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:37:13.498195 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4e7565ce-0e48-49e6-be1b-8d8761560051 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:13.502361 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4e7565ce-0e48-49e6-be1b-8d8761560051 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:13.502361 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4e7565ce-0e48-49e6-be1b-8d8761560051 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:13.502361 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4e7565ce-0e48-49e6-be1b-8d8761560051 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:13.502361 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4e7565ce-0e48-49e6-be1b-8d8761560051 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:13.502784 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4e7565ce-0e48-49e6-be1b-8d8761560051 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:13.639936 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4e7565ce-0e48-49e6-be1b-8d8761560051 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:13.648418 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4e7565ce-0e48-49e6-be1b-8d8761560051 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1577 microversion: 2.1 time: 0.162325 Aug 21 01:37:13.649046 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 544/1071] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:13 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1577 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:14.673350 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-cfa28568-ee90-45e5-b03e-3059f74f5a23 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:14.674336 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-cfa28568-ee90-45e5-b03e-3059f74f5a23 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:14.681462 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cfa28568-ee90-45e5-b03e-3059f74f5a23 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:14.683497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cfa28568-ee90-45e5-b03e-3059f74f5a23 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:14.683497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cfa28568-ee90-45e5-b03e-3059f74f5a23 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:14.683497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cfa28568-ee90-45e5-b03e-3059f74f5a23 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:14.683497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cfa28568-ee90-45e5-b03e-3059f74f5a23 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:14.683978 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-cfa28568-ee90-45e5-b03e-3059f74f5a23 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:14.821259 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-cfa28568-ee90-45e5-b03e-3059f74f5a23 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:14.829165 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-cfa28568-ee90-45e5-b03e-3059f74f5a23 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1577 microversion: 2.1 time: 0.163232 Aug 21 01:37:14.829719 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 528/1072] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:14 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1577 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:15.851471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-abcd8af1-fd92-451f-8240-2562a8c049f3 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:15.852290 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-abcd8af1-fd92-451f-8240-2562a8c049f3 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:15.859060 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-abcd8af1-fd92-451f-8240-2562a8c049f3 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:15.859545 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-abcd8af1-fd92-451f-8240-2562a8c049f3 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:15.859978 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-abcd8af1-fd92-451f-8240-2562a8c049f3 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:15.860705 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-abcd8af1-fd92-451f-8240-2562a8c049f3 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:15.861158 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-abcd8af1-fd92-451f-8240-2562a8c049f3 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:15.861580 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-abcd8af1-fd92-451f-8240-2562a8c049f3 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:15.993691 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-abcd8af1-fd92-451f-8240-2562a8c049f3 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:16.002100 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-abcd8af1-fd92-451f-8240-2562a8c049f3 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1577 microversion: 2.1 time: 0.155169 Aug 21 01:37:16.002661 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 545/1073] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:15 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1577 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:16.997402 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-a154cb4c-adf6-4a35-9cfd-0e1b8f93d3ac req-68de3c9f-a598-49fd-b424-2e5ccf0268cf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "85c51937-c541-455e-a0db-4d06041c67be", "name": "network-vif-deleted", "tag": "b8bc8818-25c9-4200-8011-9f19e85c6eec"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:37:17.008133 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-a154cb4c-adf6-4a35-9cfd-0e1b8f93d3ac req-68de3c9f-a598-49fd-b424-2e5ccf0268cf service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:17.008635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-a154cb4c-adf6-4a35-9cfd-0e1b8f93d3ac req-68de3c9f-a598-49fd-b424-2e5ccf0268cf service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:17.009066 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-a154cb4c-adf6-4a35-9cfd-0e1b8f93d3ac req-68de3c9f-a598-49fd-b424-2e5ccf0268cf service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:17.024941 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-a154cb4c-adf6-4a35-9cfd-0e1b8f93d3ac req-68de3c9f-a598-49fd-b424-2e5ccf0268cf service nova] Creating event network-vif-deleted:b8bc8818-25c9-4200-8011-9f19e85c6eec for instance 85c51937-c541-455e-a0db-4d06041c67be on hv-cinder-maste Aug 21 01:37:17.026402 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2f38c8c8-2c97-4012-a419-71cb3e2a54c0 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:17.027105 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-2f38c8c8-2c97-4012-a419-71cb3e2a54c0 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:17.031967 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-a154cb4c-adf6-4a35-9cfd-0e1b8f93d3ac req-68de3c9f-a598-49fd-b424-2e5ccf0268cf service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038106 Aug 21 01:37:17.033955 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2f38c8c8-2c97-4012-a419-71cb3e2a54c0 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:17.034384 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 529/1074] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:37:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:37:17.037565 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2f38c8c8-2c97-4012-a419-71cb3e2a54c0 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:17.038054 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2f38c8c8-2c97-4012-a419-71cb3e2a54c0 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:17.039610 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2f38c8c8-2c97-4012-a419-71cb3e2a54c0 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:17.040067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2f38c8c8-2c97-4012-a419-71cb3e2a54c0 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:17.040523 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2f38c8c8-2c97-4012-a419-71cb3e2a54c0 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:17.109129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-2f38c8c8-2c97-4012-a419-71cb3e2a54c0 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:17.119869 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2f38c8c8-2c97-4012-a419-71cb3e2a54c0 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1509 microversion: 2.1 time: 0.099763 Aug 21 01:37:17.120880 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 546/1075] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:17 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1509 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:18.140711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:18.141548 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:18.148688 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:18.149406 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:18.149824 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:18.150519 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:18.150973 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:18.151398 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:18.186162 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:18.186647 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:18.187081 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:18.187525 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lazy-loading 'fault' on Instance uuid 85c51937-c541-455e-a0db-4d06041c67be {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:37:18.243241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:18.251967 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-55508481-3c91-499f-96fb-ecc4d404258b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 200 len: 1229 microversion: 2.1 time: 0.115402 Aug 21 01:37:18.252940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 530/1076] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:18 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 1229 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:37:19.272689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-49bb1d03-b628-4f92-8767-73a2fa8d4a0a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:19.273305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-49bb1d03-b628-4f92-8767-73a2fa8d4a0a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:19.279862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-49bb1d03-b628-4f92-8767-73a2fa8d4a0a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:19.280363 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-49bb1d03-b628-4f92-8767-73a2fa8d4a0a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:19.280808 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-49bb1d03-b628-4f92-8767-73a2fa8d4a0a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:19.281506 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-49bb1d03-b628-4f92-8767-73a2fa8d4a0a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:19.281961 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-49bb1d03-b628-4f92-8767-73a2fa8d4a0a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:19.282389 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-49bb1d03-b628-4f92-8767-73a2fa8d4a0a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:19.296207 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-49bb1d03-b628-4f92-8767-73a2fa8d4a0a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] HTTP exception thrown: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-49bb1d03-b628-4f92-8767-73a2fa8d4a0a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Request method failure captured: Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be HTTP/1.1 Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYufRkMP3i_c3e0Vr_zgnzF_eSwVsk8M2N0eQOxptmM6V3DchJQy0M67Fr9G61C8FQolpg4xfXTwdedQKOqV9nQf-UOv9V2jyiY57zjWmPWe7uvA2VzMRuLDZd_CPbp7XgOPBjzm_Eiw1Dk-g2GXgEtB0MrihdzJ7mHtU79rwyzdn7A Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 4124766f4f2843229059dbbdf7a2de74 Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:37:19.297737 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:37:19.299476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:37:19.299476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:37:19.299476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 4124766f4f2843229059dbbdf7a2de74 Aug 21 01:37:19.299476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:37:19.299476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-TestVolumeBootPattern-2085187589-project Aug 21 01:37:19.299476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:37:19.299476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:37:19.299476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: fef6461d7ca14cab829412ce3b53852a Aug 21 01:37:19.299476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-TestVolumeBootPattern-2085187589-project Aug 21 01:37:19.299476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:37:19.299476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:37:19.299476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '85c51937-c541-455e-a0db-4d06041c67be'} Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:37:19.300568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:37:19.302861 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:37:19.304391 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-49bb1d03-b628-4f92-8767-73a2fa8d4a0a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Returning 404 to user: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:37:19.304391 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-49bb1d03-b628-4f92-8767-73a2fa8d4a0a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 404 len: 111 microversion: 2.1 time: 0.030394 Aug 21 01:37:19.304391 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 547/1077] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:19 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:37:19.399691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-552844390", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1465137419", "security_groups": [{"name": "tempest-secgroup-smoke-1796196116"}], "block_device_mapping_v2": [{"uuid": "b1308b3c-c495-472c-b9cb-85570057d482", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "e4ca6005-16bf-4430-905e-1ccbd3a7f5d3"}]}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:37:19.514322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] validate_networks() for [('e4ca6005-16bf-4430-905e-1ccbd3a7f5d3', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:37:20.061923 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:37:20.062349 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:37:20.062849 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:37:20.063383 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:37:20.063702 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:37:20.073098 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:20.073578 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:20.074010 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:20.074748 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:20.075208 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:20.075634 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:20.094147 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Getting quotas for project 4124766f4f2843229059dbbdf7a2de74. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:37:20.098380 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Getting quotas for user fef6461d7ca14cab829412ce3b53852a and project 4124766f4f2843229059dbbdf7a2de74. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:37:20.157781 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:37:20.159878 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b1308b3c-c495-472c-b9cb-85570057d482',volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:37:20.563361 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2c205ee3-3681-49ac-b44f-0587071c574a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.167583 Aug 21 01:37:20.564316 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 531/1078] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:37:19 2022] POST /compute/v2.1/servers => generated 406 bytes in 1169 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:37:20.581984 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-28cf7930-34dc-4ba2-90f6-75afd14b4039 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:20.582607 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-28cf7930-34dc-4ba2-90f6-75afd14b4039 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:20.637216 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-28cf7930-34dc-4ba2-90f6-75afd14b4039 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:20.644045 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-28cf7930-34dc-4ba2-90f6-75afd14b4039 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1094 microversion: 2.1 time: 0.067102 Aug 21 01:37:20.644850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 548/1079] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:20 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1094 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:21.664793 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a1de6d2b-6cff-4010-991e-3ec30bb40870 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:21.666000 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a1de6d2b-6cff-4010-991e-3ec30bb40870 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:21.672181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a1de6d2b-6cff-4010-991e-3ec30bb40870 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:21.672684 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a1de6d2b-6cff-4010-991e-3ec30bb40870 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:21.673125 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a1de6d2b-6cff-4010-991e-3ec30bb40870 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:21.673820 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a1de6d2b-6cff-4010-991e-3ec30bb40870 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:21.674314 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a1de6d2b-6cff-4010-991e-3ec30bb40870 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:21.674742 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a1de6d2b-6cff-4010-991e-3ec30bb40870 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:21.740766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a1de6d2b-6cff-4010-991e-3ec30bb40870 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:21.748735 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a1de6d2b-6cff-4010-991e-3ec30bb40870 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1144 microversion: 2.1 time: 0.088335 Aug 21 01:37:21.750011 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 532/1080] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:21 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1144 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:22.769088 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-42d0a46e-6c63-47de-8be3-2171973b4bf5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:22.769962 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-42d0a46e-6c63-47de-8be3-2171973b4bf5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:22.777692 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-42d0a46e-6c63-47de-8be3-2171973b4bf5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:22.778181 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-42d0a46e-6c63-47de-8be3-2171973b4bf5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:22.778624 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-42d0a46e-6c63-47de-8be3-2171973b4bf5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:22.779321 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-42d0a46e-6c63-47de-8be3-2171973b4bf5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:22.779780 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-42d0a46e-6c63-47de-8be3-2171973b4bf5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:22.780232 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-42d0a46e-6c63-47de-8be3-2171973b4bf5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:23.139279 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-42d0a46e-6c63-47de-8be3-2171973b4bf5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:23.147217 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-42d0a46e-6c63-47de-8be3-2171973b4bf5 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1200 microversion: 2.1 time: 0.382239 Aug 21 01:37:23.147786 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 549/1081] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:22 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1200 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:24.170932 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-412b0b22-9eef-4cd8-ab22-81e34b40a8ab tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:24.172549 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-412b0b22-9eef-4cd8-ab22-81e34b40a8ab tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:24.185482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-412b0b22-9eef-4cd8-ab22-81e34b40a8ab tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:24.185482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-412b0b22-9eef-4cd8-ab22-81e34b40a8ab tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:24.185482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-412b0b22-9eef-4cd8-ab22-81e34b40a8ab tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:24.185482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-412b0b22-9eef-4cd8-ab22-81e34b40a8ab tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:24.185918 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-412b0b22-9eef-4cd8-ab22-81e34b40a8ab tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:24.185918 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-412b0b22-9eef-4cd8-ab22-81e34b40a8ab tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:24.414391 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-412b0b22-9eef-4cd8-ab22-81e34b40a8ab tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:24.423435 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-412b0b22-9eef-4cd8-ab22-81e34b40a8ab tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1278 microversion: 2.1 time: 0.258194 Aug 21 01:37:24.424203 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 533/1082] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:24 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1278 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:24.629619 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-9664732f-3d0d-4b99-9d93-06aa4da64fc8 req-f74f6cb0-cf6a-4f5f-9382-11dc92fe218e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb", "tag": "6fc060b4-65bc-403a-be6a-d0408f800a0a"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:37:24.639995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9664732f-3d0d-4b99-9d93-06aa4da64fc8 req-f74f6cb0-cf6a-4f5f-9382-11dc92fe218e service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:24.640491 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9664732f-3d0d-4b99-9d93-06aa4da64fc8 req-f74f6cb0-cf6a-4f5f-9382-11dc92fe218e service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:24.640935 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-9664732f-3d0d-4b99-9d93-06aa4da64fc8 req-f74f6cb0-cf6a-4f5f-9382-11dc92fe218e service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:24.656508 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-9664732f-3d0d-4b99-9d93-06aa4da64fc8 req-f74f6cb0-cf6a-4f5f-9382-11dc92fe218e service nova] Creating event network-changed:6fc060b4-65bc-403a-be6a-d0408f800a0a for instance 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb on hv-cinder-maste Aug 21 01:37:24.663237 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-9664732f-3d0d-4b99-9d93-06aa4da64fc8 req-f74f6cb0-cf6a-4f5f-9382-11dc92fe218e service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037195 Aug 21 01:37:24.664095 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 550/1083] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:37:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:37:25.456167 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2dae4e08-1b4d-41b1-9cb6-043b08d1023a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:25.456792 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2dae4e08-1b4d-41b1-9cb6-043b08d1023a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:25.464349 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2dae4e08-1b4d-41b1-9cb6-043b08d1023a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:25.464839 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2dae4e08-1b4d-41b1-9cb6-043b08d1023a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:25.465280 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2dae4e08-1b4d-41b1-9cb6-043b08d1023a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:25.465976 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2dae4e08-1b4d-41b1-9cb6-043b08d1023a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:25.466433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2dae4e08-1b4d-41b1-9cb6-043b08d1023a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:25.466864 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2dae4e08-1b4d-41b1-9cb6-043b08d1023a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:25.604947 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2dae4e08-1b4d-41b1-9cb6-043b08d1023a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:25.612743 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2dae4e08-1b4d-41b1-9cb6-043b08d1023a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1266 microversion: 2.1 time: 0.160619 Aug 21 01:37:25.613307 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 534/1084] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:25 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1266 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:26.635490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-a29777a8-a505-4b78-923f-03b30c0019c4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:26.636177 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-a29777a8-a505-4b78-923f-03b30c0019c4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:26.643142 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a29777a8-a505-4b78-923f-03b30c0019c4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:26.643663 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a29777a8-a505-4b78-923f-03b30c0019c4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:26.644129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a29777a8-a505-4b78-923f-03b30c0019c4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:26.645316 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a29777a8-a505-4b78-923f-03b30c0019c4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:26.645776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a29777a8-a505-4b78-923f-03b30c0019c4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:26.646202 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-a29777a8-a505-4b78-923f-03b30c0019c4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:26.778443 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-a29777a8-a505-4b78-923f-03b30c0019c4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:26.786101 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-a29777a8-a505-4b78-923f-03b30c0019c4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1266 microversion: 2.1 time: 0.155452 Aug 21 01:37:26.786665 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 551/1085] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:26 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1266 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:27.807143 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e1fad9e0-9793-48c4-9836-648f85aa8592 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:27.807762 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e1fad9e0-9793-48c4-9836-648f85aa8592 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:27.813945 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e1fad9e0-9793-48c4-9836-648f85aa8592 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:27.814433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e1fad9e0-9793-48c4-9836-648f85aa8592 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:27.814870 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e1fad9e0-9793-48c4-9836-648f85aa8592 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:27.815565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e1fad9e0-9793-48c4-9836-648f85aa8592 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:27.816052 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e1fad9e0-9793-48c4-9836-648f85aa8592 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:27.816505 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e1fad9e0-9793-48c4-9836-648f85aa8592 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:27.953893 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e1fad9e0-9793-48c4-9836-648f85aa8592 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:27.962458 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e1fad9e0-9793-48c4-9836-648f85aa8592 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1266 microversion: 2.1 time: 0.159211 Aug 21 01:37:27.963014 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 535/1086] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:27 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1266 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:28.983431 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-cdce9968-3337-4ad9-8597-05ede4fa5743 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:28.984042 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-cdce9968-3337-4ad9-8597-05ede4fa5743 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:28.991277 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cdce9968-3337-4ad9-8597-05ede4fa5743 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:28.991760 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cdce9968-3337-4ad9-8597-05ede4fa5743 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:28.992244 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cdce9968-3337-4ad9-8597-05ede4fa5743 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:28.992936 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cdce9968-3337-4ad9-8597-05ede4fa5743 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:28.993393 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cdce9968-3337-4ad9-8597-05ede4fa5743 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:28.993818 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-cdce9968-3337-4ad9-8597-05ede4fa5743 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:29.125860 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-cdce9968-3337-4ad9-8597-05ede4fa5743 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:29.133178 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-cdce9968-3337-4ad9-8597-05ede4fa5743 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1266 microversion: 2.1 time: 0.153558 Aug 21 01:37:29.133765 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 552/1087] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:28 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1266 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:30.153699 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6055d810-158a-4a8b-852c-950332c0099b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:30.154237 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6055d810-158a-4a8b-852c-950332c0099b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:30.156737 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:37:30.157401 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=6 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:37:30.162281 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6055d810-158a-4a8b-852c-950332c0099b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:30.162764 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6055d810-158a-4a8b-852c-950332c0099b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:30.163197 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6055d810-158a-4a8b-852c-950332c0099b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:30.163893 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6055d810-158a-4a8b-852c-950332c0099b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:30.164404 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6055d810-158a-4a8b-852c-950332c0099b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:30.164787 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6055d810-158a-4a8b-852c-950332c0099b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:30.310809 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6055d810-158a-4a8b-852c-950332c0099b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:30.319255 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6055d810-158a-4a8b-852c-950332c0099b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1266 microversion: 2.1 time: 0.169537 Aug 21 01:37:30.319827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 536/1088] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:30 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1266 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:31.340490 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3de14ff3-a9a4-47e2-ac13-5f311472deca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:31.341106 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3de14ff3-a9a4-47e2-ac13-5f311472deca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:31.343456 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:37:31.348343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3de14ff3-a9a4-47e2-ac13-5f311472deca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:31.348829 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3de14ff3-a9a4-47e2-ac13-5f311472deca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:31.349263 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3de14ff3-a9a4-47e2-ac13-5f311472deca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:31.349952 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3de14ff3-a9a4-47e2-ac13-5f311472deca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:31.350589 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3de14ff3-a9a4-47e2-ac13-5f311472deca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:31.350811 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3de14ff3-a9a4-47e2-ac13-5f311472deca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:31.485289 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3de14ff3-a9a4-47e2-ac13-5f311472deca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:31.494338 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3de14ff3-a9a4-47e2-ac13-5f311472deca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1266 microversion: 2.1 time: 0.157845 Aug 21 01:37:31.494918 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 553/1089] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:31 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1266 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:32.515038 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2cb9b46f-f047-4903-a5cd-a9ef13ce6a87 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:32.515722 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2cb9b46f-f047-4903-a5cd-a9ef13ce6a87 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:32.523370 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2cb9b46f-f047-4903-a5cd-a9ef13ce6a87 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:32.523855 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2cb9b46f-f047-4903-a5cd-a9ef13ce6a87 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:32.524318 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2cb9b46f-f047-4903-a5cd-a9ef13ce6a87 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:32.525016 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2cb9b46f-f047-4903-a5cd-a9ef13ce6a87 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:32.525467 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2cb9b46f-f047-4903-a5cd-a9ef13ce6a87 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:32.525887 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2cb9b46f-f047-4903-a5cd-a9ef13ce6a87 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:32.665000 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2cb9b46f-f047-4903-a5cd-a9ef13ce6a87 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:32.672796 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2cb9b46f-f047-4903-a5cd-a9ef13ce6a87 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1266 microversion: 2.1 time: 0.161829 Aug 21 01:37:32.673356 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 537/1090] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:32 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:33.693705 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e060e5c5-a6ae-44c7-bfe7-79177007a877 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:33.694322 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e060e5c5-a6ae-44c7-bfe7-79177007a877 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:33.701341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e060e5c5-a6ae-44c7-bfe7-79177007a877 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:33.701834 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e060e5c5-a6ae-44c7-bfe7-79177007a877 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:33.702277 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e060e5c5-a6ae-44c7-bfe7-79177007a877 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:33.705204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e060e5c5-a6ae-44c7-bfe7-79177007a877 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:33.705204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e060e5c5-a6ae-44c7-bfe7-79177007a877 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:33.705204 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e060e5c5-a6ae-44c7-bfe7-79177007a877 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:33.853053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e060e5c5-a6ae-44c7-bfe7-79177007a877 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:33.865883 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e060e5c5-a6ae-44c7-bfe7-79177007a877 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1266 microversion: 2.1 time: 0.171955 Aug 21 01:37:33.865883 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 554/1091] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:33 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1266 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:34.882927 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2199d70c-3d50-4940-a27f-da6360c26744 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:34.883637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2199d70c-3d50-4940-a27f-da6360c26744 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:34.889637 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2199d70c-3d50-4940-a27f-da6360c26744 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:34.890131 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2199d70c-3d50-4940-a27f-da6360c26744 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:34.890562 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2199d70c-3d50-4940-a27f-da6360c26744 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:34.891264 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2199d70c-3d50-4940-a27f-da6360c26744 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:34.891717 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2199d70c-3d50-4940-a27f-da6360c26744 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:34.892171 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2199d70c-3d50-4940-a27f-da6360c26744 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:35.032199 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2199d70c-3d50-4940-a27f-da6360c26744 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:35.040555 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2199d70c-3d50-4940-a27f-da6360c26744 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1266 microversion: 2.1 time: 0.162228 Aug 21 01:37:35.041111 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 538/1092] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:34 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:35.474589 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-ea15c398-dae6-45c2-8c1b-c4dc0977067c req-b9df22b0-39d3-4c5b-a098-0043f3ed1b2d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb", "name": "network-vif-plugged", "status": "completed", "tag": "6fc060b4-65bc-403a-be6a-d0408f800a0a"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:37:35.485675 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ea15c398-dae6-45c2-8c1b-c4dc0977067c req-b9df22b0-39d3-4c5b-a098-0043f3ed1b2d service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:35.486295 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ea15c398-dae6-45c2-8c1b-c4dc0977067c req-b9df22b0-39d3-4c5b-a098-0043f3ed1b2d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:35.486799 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-ea15c398-dae6-45c2-8c1b-c4dc0977067c req-b9df22b0-39d3-4c5b-a098-0043f3ed1b2d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:35.499291 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-ea15c398-dae6-45c2-8c1b-c4dc0977067c req-b9df22b0-39d3-4c5b-a098-0043f3ed1b2d service nova] Creating event network-vif-plugged:6fc060b4-65bc-403a-be6a-d0408f800a0a for instance 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb on hv-cinder-maste Aug 21 01:37:35.504445 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-ea15c398-dae6-45c2-8c1b-c4dc0977067c req-b9df22b0-39d3-4c5b-a098-0043f3ed1b2d service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.158953 Aug 21 01:37:35.505049 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 555/1093] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:37:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:37:36.061628 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-df4ef32a-52e3-4d0c-b9be-1f2873383ef6 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:36.062231 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-df4ef32a-52e3-4d0c-b9be-1f2873383ef6 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:36.069954 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-df4ef32a-52e3-4d0c-b9be-1f2873383ef6 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:36.070450 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-df4ef32a-52e3-4d0c-b9be-1f2873383ef6 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:36.070903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-df4ef32a-52e3-4d0c-b9be-1f2873383ef6 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:36.071612 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-df4ef32a-52e3-4d0c-b9be-1f2873383ef6 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:36.072068 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-df4ef32a-52e3-4d0c-b9be-1f2873383ef6 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:36.072523 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-df4ef32a-52e3-4d0c-b9be-1f2873383ef6 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:36.209676 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-df4ef32a-52e3-4d0c-b9be-1f2873383ef6 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:36.217482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-df4ef32a-52e3-4d0c-b9be-1f2873383ef6 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1266 microversion: 2.1 time: 0.159679 Aug 21 01:37:36.218140 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 539/1094] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:36 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1266 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:37.239417 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-68cccace-7e7b-4268-b35d-fc68b2a6ce82 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:37.240033 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-68cccace-7e7b-4268-b35d-fc68b2a6ce82 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:37.246205 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68cccace-7e7b-4268-b35d-fc68b2a6ce82 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:37.251085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68cccace-7e7b-4268-b35d-fc68b2a6ce82 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:37.251085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68cccace-7e7b-4268-b35d-fc68b2a6ce82 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:37.251085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68cccace-7e7b-4268-b35d-fc68b2a6ce82 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:37.251085 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68cccace-7e7b-4268-b35d-fc68b2a6ce82 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:37.251475 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68cccace-7e7b-4268-b35d-fc68b2a6ce82 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:37.393731 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-68cccace-7e7b-4268-b35d-fc68b2a6ce82 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:37.407181 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-68cccace-7e7b-4268-b35d-fc68b2a6ce82 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1452 microversion: 2.1 time: 0.171145 Aug 21 01:37:37.407181 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 556/1095] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:37 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1452 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:37.426414 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-6885a711-6ca2-4b52-869f-1423ed8611ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:37:37.427050 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-6885a711-6ca2-4b52-869f-1423ed8611ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:37:37.433593 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6885a711-6ca2-4b52-869f-1423ed8611ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:37.434086 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6885a711-6ca2-4b52-869f-1423ed8611ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:37.434527 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6885a711-6ca2-4b52-869f-1423ed8611ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:37.435236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6885a711-6ca2-4b52-869f-1423ed8611ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:37.435694 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6885a711-6ca2-4b52-869f-1423ed8611ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:37.436138 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-6885a711-6ca2-4b52-869f-1423ed8611ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:37.584497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-6885a711-6ca2-4b52-869f-1423ed8611ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:37:37.593692 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-6885a711-6ca2-4b52-869f-1423ed8611ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1452 microversion: 2.1 time: 0.171578 Aug 21 01:37:37.594277 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 540/1096] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:37:37 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1452 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:37:39.678142 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-7d93ab0c-ec7d-4340-a7c7-aa14955fc501 req-cc5af304-aed7-4252-b0a0-5e0ed8945cc9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb", "tag": "6fc060b4-65bc-403a-be6a-d0408f800a0a"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:37:39.687581 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-7d93ab0c-ec7d-4340-a7c7-aa14955fc501 req-cc5af304-aed7-4252-b0a0-5e0ed8945cc9 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:37:39.688444 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-7d93ab0c-ec7d-4340-a7c7-aa14955fc501 req-cc5af304-aed7-4252-b0a0-5e0ed8945cc9 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:37:39.689084 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-7d93ab0c-ec7d-4340-a7c7-aa14955fc501 req-cc5af304-aed7-4252-b0a0-5e0ed8945cc9 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:37:39.711160 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-7d93ab0c-ec7d-4340-a7c7-aa14955fc501 req-cc5af304-aed7-4252-b0a0-5e0ed8945cc9 service nova] Creating event network-changed:6fc060b4-65bc-403a-be6a-d0408f800a0a for instance 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb on hv-cinder-maste Aug 21 01:37:39.723058 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-7d93ab0c-ec7d-4340-a7c7-aa14955fc501 req-cc5af304-aed7-4252-b0a0-5e0ed8945cc9 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.166003 Aug 21 01:37:39.723058 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 557/1097] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:37:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:37:57.718109 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:37:57.718624 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Sun Aug 21 01:37:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:38:03.637859 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-424104679", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "networks": [{"uuid": "0e2a927d-deb9-4756-b3d9-f933a82be077"}]}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:38:03.646506 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=48,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:03.647172 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=86,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:03.696626 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] validate_networks() for [('0e2a927d-deb9-4756-b3d9-f933a82be077', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:38:04.041394 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:38:04.041812 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:38:04.042283 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:38:04.042685 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:38:04.043094 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:38:04.051605 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:04.052089 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:04.053114 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:04.053774 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:04.054221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:04.054640 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:04.071332 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Getting quotas for project 5d895d8b54ac4d23aaca3e1065b6035f. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:38:04.075107 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Getting quotas for user 08e7e7eadd5846d29c28e6e3b5c5d76f and project 5d895d8b54ac4d23aaca3e1065b6035f. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:38:04.086015 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:38:04.088198 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:38:04.218487 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-54feb871-fde1-4da6-be30-3bc973af680e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.584491 Aug 21 01:38:04.219305 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 541/1098] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:38:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 586 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:38:04.228393 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=30 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:04.229171 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=62 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:04.233463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-cf5edcd4-8f64-48fc-b5d3-73b8ef53139a tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:04.234168 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-cf5edcd4-8f64-48fc-b5d3-73b8ef53139a tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:04.290595 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-cf5edcd4-8f64-48fc-b5d3-73b8ef53139a tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:04.299530 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-cf5edcd4-8f64-48fc-b5d3-73b8ef53139a tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1213 microversion: 2.1 time: 0.072679 Aug 21 01:38:04.300342 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 558/1099] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:04 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1213 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:05.321886 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-772e3c01-21d5-47c3-9188-73e3c2d9c5af tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:05.322535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-772e3c01-21d5-47c3-9188-73e3c2d9c5af tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:05.329411 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-772e3c01-21d5-47c3-9188-73e3c2d9c5af tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:05.329894 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-772e3c01-21d5-47c3-9188-73e3c2d9c5af tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:05.330335 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-772e3c01-21d5-47c3-9188-73e3c2d9c5af tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:05.331033 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-772e3c01-21d5-47c3-9188-73e3c2d9c5af tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:05.331492 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-772e3c01-21d5-47c3-9188-73e3c2d9c5af tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:05.331917 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-772e3c01-21d5-47c3-9188-73e3c2d9c5af tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:05.393587 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-772e3c01-21d5-47c3-9188-73e3c2d9c5af tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:05.401282 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-772e3c01-21d5-47c3-9188-73e3c2d9c5af tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1217 microversion: 2.1 time: 0.084551 Aug 21 01:38:05.401847 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 542/1100] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:05 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1217 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:06.421230 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-31daa92a-0353-48e2-b783-19bb228b668e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:06.421862 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-31daa92a-0353-48e2-b783-19bb228b668e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:06.430156 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31daa92a-0353-48e2-b783-19bb228b668e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:06.439850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31daa92a-0353-48e2-b783-19bb228b668e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:06.439850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31daa92a-0353-48e2-b783-19bb228b668e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:06.439850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31daa92a-0353-48e2-b783-19bb228b668e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:06.439850 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31daa92a-0353-48e2-b783-19bb228b668e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:06.440343 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-31daa92a-0353-48e2-b783-19bb228b668e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:06.507501 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-31daa92a-0353-48e2-b783-19bb228b668e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:06.515904 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-31daa92a-0353-48e2-b783-19bb228b668e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1273 microversion: 2.1 time: 0.098565 Aug 21 01:38:06.528455 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 559/1101] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:06 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1273 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:07.539406 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1b5a6770-49f2-4899-8bfb-c6fe3fabbaf3 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:07.540044 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1b5a6770-49f2-4899-8bfb-c6fe3fabbaf3 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:07.546728 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1b5a6770-49f2-4899-8bfb-c6fe3fabbaf3 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:07.550421 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1b5a6770-49f2-4899-8bfb-c6fe3fabbaf3 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:07.550421 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1b5a6770-49f2-4899-8bfb-c6fe3fabbaf3 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:07.550421 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1b5a6770-49f2-4899-8bfb-c6fe3fabbaf3 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:07.550421 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1b5a6770-49f2-4899-8bfb-c6fe3fabbaf3 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:07.550889 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1b5a6770-49f2-4899-8bfb-c6fe3fabbaf3 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:07.741994 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-1b5a6770-49f2-4899-8bfb-c6fe3fabbaf3 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:07.749684 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1b5a6770-49f2-4899-8bfb-c6fe3fabbaf3 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1313 microversion: 2.1 time: 0.214801 Aug 21 01:38:07.751604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 543/1102] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:07 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1313 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:08.180531 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-94a54ba5-41a3-42a7-97d3-43da76cfaf32 req-2e9eb356-e566-48c6-9455-73b40e5744c5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "146ae855-7a33-4572-ad09-b54ba7de0bcc", "tag": "4c56ba2b-c08c-48d3-a24b-3e57d0363175"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:38:08.191796 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-94a54ba5-41a3-42a7-97d3-43da76cfaf32 req-2e9eb356-e566-48c6-9455-73b40e5744c5 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:08.192329 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-94a54ba5-41a3-42a7-97d3-43da76cfaf32 req-2e9eb356-e566-48c6-9455-73b40e5744c5 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:08.192852 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-94a54ba5-41a3-42a7-97d3-43da76cfaf32 req-2e9eb356-e566-48c6-9455-73b40e5744c5 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:08.205534 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-94a54ba5-41a3-42a7-97d3-43da76cfaf32 req-2e9eb356-e566-48c6-9455-73b40e5744c5 service nova] Creating event network-changed:4c56ba2b-c08c-48d3-a24b-3e57d0363175 for instance 146ae855-7a33-4572-ad09-b54ba7de0bcc on hv-cinder-maste Aug 21 01:38:08.211565 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-94a54ba5-41a3-42a7-97d3-43da76cfaf32 req-2e9eb356-e566-48c6-9455-73b40e5744c5 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034818 Aug 21 01:38:08.212270 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 560/1103] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:38:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:38:08.770887 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-065545c0-48b4-4c20-9e47-e7125733d925 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:08.771496 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-065545c0-48b4-4c20-9e47-e7125733d925 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:08.778475 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-065545c0-48b4-4c20-9e47-e7125733d925 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:08.779044 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-065545c0-48b4-4c20-9e47-e7125733d925 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:08.779516 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-065545c0-48b4-4c20-9e47-e7125733d925 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:08.780255 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-065545c0-48b4-4c20-9e47-e7125733d925 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:08.780710 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-065545c0-48b4-4c20-9e47-e7125733d925 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:08.781137 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-065545c0-48b4-4c20-9e47-e7125733d925 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:09.141479 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-065545c0-48b4-4c20-9e47-e7125733d925 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:09.149950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-065545c0-48b4-4c20-9e47-e7125733d925 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1313 microversion: 2.1 time: 0.382967 Aug 21 01:38:09.150524 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 544/1104] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:08 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1313 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:10.172925 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-7f39c2b6-fa7f-4c43-969f-d5d5241ec138 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:10.173627 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-7f39c2b6-fa7f-4c43-969f-d5d5241ec138 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:10.180405 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f39c2b6-fa7f-4c43-969f-d5d5241ec138 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:10.180892 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f39c2b6-fa7f-4c43-969f-d5d5241ec138 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:10.181368 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f39c2b6-fa7f-4c43-969f-d5d5241ec138 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:10.182063 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f39c2b6-fa7f-4c43-969f-d5d5241ec138 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:10.182517 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f39c2b6-fa7f-4c43-969f-d5d5241ec138 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:10.182954 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-7f39c2b6-fa7f-4c43-969f-d5d5241ec138 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:10.314757 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-7f39c2b6-fa7f-4c43-969f-d5d5241ec138 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:10.322283 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-7f39c2b6-fa7f-4c43-969f-d5d5241ec138 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1313 microversion: 2.1 time: 0.154258 Aug 21 01:38:10.322929 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 561/1105] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:10 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1313 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:11.346199 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-bde21c6c-7b23-48bd-9399-67ce2f8f03b8 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:11.346802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-bde21c6c-7b23-48bd-9399-67ce2f8f03b8 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:11.353295 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bde21c6c-7b23-48bd-9399-67ce2f8f03b8 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:11.353779 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bde21c6c-7b23-48bd-9399-67ce2f8f03b8 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:11.354222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bde21c6c-7b23-48bd-9399-67ce2f8f03b8 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:11.354919 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bde21c6c-7b23-48bd-9399-67ce2f8f03b8 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:11.355373 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bde21c6c-7b23-48bd-9399-67ce2f8f03b8 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:11.355797 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-bde21c6c-7b23-48bd-9399-67ce2f8f03b8 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:11.484541 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-bde21c6c-7b23-48bd-9399-67ce2f8f03b8 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:11.492709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-bde21c6c-7b23-48bd-9399-67ce2f8f03b8 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1313 microversion: 2.1 time: 0.150424 Aug 21 01:38:11.493265 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 545/1106] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:11 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1313 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:12.513265 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-27c8d93a-3943-4fc9-86b1-88f932cb6875 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:12.513810 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-27c8d93a-3943-4fc9-86b1-88f932cb6875 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:12.520535 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-27c8d93a-3943-4fc9-86b1-88f932cb6875 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:12.523341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-27c8d93a-3943-4fc9-86b1-88f932cb6875 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:12.523341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-27c8d93a-3943-4fc9-86b1-88f932cb6875 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:12.523341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-27c8d93a-3943-4fc9-86b1-88f932cb6875 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:12.523341 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-27c8d93a-3943-4fc9-86b1-88f932cb6875 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:12.523727 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-27c8d93a-3943-4fc9-86b1-88f932cb6875 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:12.652735 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-27c8d93a-3943-4fc9-86b1-88f932cb6875 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:12.660424 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-27c8d93a-3943-4fc9-86b1-88f932cb6875 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1313 microversion: 2.1 time: 0.151192 Aug 21 01:38:12.660990 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 562/1107] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:12 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1313 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:13.682259 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f42209db-9583-49c4-9716-835cb6754e38 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:13.682872 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f42209db-9583-49c4-9716-835cb6754e38 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:13.689604 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f42209db-9583-49c4-9716-835cb6754e38 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:13.690091 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f42209db-9583-49c4-9716-835cb6754e38 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:13.690531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f42209db-9583-49c4-9716-835cb6754e38 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:13.691355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f42209db-9583-49c4-9716-835cb6754e38 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:13.691685 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f42209db-9583-49c4-9716-835cb6754e38 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:13.692122 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f42209db-9583-49c4-9716-835cb6754e38 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:13.834619 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f42209db-9583-49c4-9716-835cb6754e38 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:13.842397 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f42209db-9583-49c4-9716-835cb6754e38 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1313 microversion: 2.1 time: 0.164543 Aug 21 01:38:13.842967 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 546/1108] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:13 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1313 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:14.862955 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-88ff727e-0fb3-4f0f-ab93-f8193667aaa9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:14.863563 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-88ff727e-0fb3-4f0f-ab93-f8193667aaa9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:14.865793 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:14.870712 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-88ff727e-0fb3-4f0f-ab93-f8193667aaa9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:14.871191 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-88ff727e-0fb3-4f0f-ab93-f8193667aaa9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:14.871739 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-88ff727e-0fb3-4f0f-ab93-f8193667aaa9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:14.872499 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-88ff727e-0fb3-4f0f-ab93-f8193667aaa9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:14.878512 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-88ff727e-0fb3-4f0f-ab93-f8193667aaa9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:14.878512 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-88ff727e-0fb3-4f0f-ab93-f8193667aaa9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:15.015327 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-88ff727e-0fb3-4f0f-ab93-f8193667aaa9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:15.024289 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-88ff727e-0fb3-4f0f-ab93-f8193667aaa9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1313 microversion: 2.1 time: 0.165191 Aug 21 01:38:15.024856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 563/1109] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:14 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1313 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:15.572542 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-5509883e-783c-438f-a89f-0948af02c292 req-19637458-efd7-479d-9a23-c58939f16bc0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "146ae855-7a33-4572-ad09-b54ba7de0bcc", "name": "network-vif-plugged", "status": "completed", "tag": "4c56ba2b-c08c-48d3-a24b-3e57d0363175"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:38:15.577516 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:15.578182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=6 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:15.583860 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-5509883e-783c-438f-a89f-0948af02c292 req-19637458-efd7-479d-9a23-c58939f16bc0 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:15.584625 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-5509883e-783c-438f-a89f-0948af02c292 req-19637458-efd7-479d-9a23-c58939f16bc0 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:15.585075 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-5509883e-783c-438f-a89f-0948af02c292 req-19637458-efd7-479d-9a23-c58939f16bc0 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:15.596055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-5509883e-783c-438f-a89f-0948af02c292 req-19637458-efd7-479d-9a23-c58939f16bc0 service nova] Creating event network-vif-plugged:4c56ba2b-c08c-48d3-a24b-3e57d0363175 for instance 146ae855-7a33-4572-ad09-b54ba7de0bcc on hv-cinder-maste Aug 21 01:38:15.601810 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-5509883e-783c-438f-a89f-0948af02c292 req-19637458-efd7-479d-9a23-c58939f16bc0 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032666 Aug 21 01:38:15.602096 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 547/1110] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:38:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:38:16.044901 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-c491f774-2b6c-4ea1-9426-fe0d909fb551 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:16.045747 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-c491f774-2b6c-4ea1-9426-fe0d909fb551 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:16.053211 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c491f774-2b6c-4ea1-9426-fe0d909fb551 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:16.053739 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c491f774-2b6c-4ea1-9426-fe0d909fb551 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:16.054259 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c491f774-2b6c-4ea1-9426-fe0d909fb551 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:16.054824 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c491f774-2b6c-4ea1-9426-fe0d909fb551 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:16.055271 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c491f774-2b6c-4ea1-9426-fe0d909fb551 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:16.055690 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-c491f774-2b6c-4ea1-9426-fe0d909fb551 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:16.188382 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-c491f774-2b6c-4ea1-9426-fe0d909fb551 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:16.196047 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-c491f774-2b6c-4ea1-9426-fe0d909fb551 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1313 microversion: 2.1 time: 0.155038 Aug 21 01:38:16.196942 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 564/1111] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:16 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1313 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:17.221237 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e54476ac-4a24-4f56-ae7e-020042d875da tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:17.221935 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e54476ac-4a24-4f56-ae7e-020042d875da tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:17.228857 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e54476ac-4a24-4f56-ae7e-020042d875da tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:17.229338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e54476ac-4a24-4f56-ae7e-020042d875da tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:17.229771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e54476ac-4a24-4f56-ae7e-020042d875da tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:17.230455 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e54476ac-4a24-4f56-ae7e-020042d875da tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:17.230944 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e54476ac-4a24-4f56-ae7e-020042d875da tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:17.231367 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e54476ac-4a24-4f56-ae7e-020042d875da tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:17.361280 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-e54476ac-4a24-4f56-ae7e-020042d875da tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:17.369671 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e54476ac-4a24-4f56-ae7e-020042d875da tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1313 microversion: 2.1 time: 0.153873 Aug 21 01:38:17.370237 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 548/1112] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:17 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1313 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:18.390785 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-3d513ed3-d9b0-41ee-9989-d8d84ceda3fd tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:18.391395 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-3d513ed3-d9b0-41ee-9989-d8d84ceda3fd tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:18.400835 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d513ed3-d9b0-41ee-9989-d8d84ceda3fd tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:18.401344 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d513ed3-d9b0-41ee-9989-d8d84ceda3fd tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:18.401785 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d513ed3-d9b0-41ee-9989-d8d84ceda3fd tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:18.402478 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d513ed3-d9b0-41ee-9989-d8d84ceda3fd tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:18.402927 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d513ed3-d9b0-41ee-9989-d8d84ceda3fd tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:18.403433 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-3d513ed3-d9b0-41ee-9989-d8d84ceda3fd tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:18.537231 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-3d513ed3-d9b0-41ee-9989-d8d84ceda3fd tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:18.545210 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-3d513ed3-d9b0-41ee-9989-d8d84ceda3fd tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1498 microversion: 2.1 time: 0.158367 Aug 21 01:38:18.545783 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 565/1113] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:18 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1498 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:19.467783 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d83e94b3-b5cd-469e-9c91-9120337fa062 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:19.468377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d83e94b3-b5cd-469e-9c91-9120337fa062 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:19.475769 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d83e94b3-b5cd-469e-9c91-9120337fa062 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:19.476324 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d83e94b3-b5cd-469e-9c91-9120337fa062 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:19.476839 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d83e94b3-b5cd-469e-9c91-9120337fa062 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:19.477554 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d83e94b3-b5cd-469e-9c91-9120337fa062 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:19.478014 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d83e94b3-b5cd-469e-9c91-9120337fa062 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:19.478452 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d83e94b3-b5cd-469e-9c91-9120337fa062 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:19.504628 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d83e94b3-b5cd-469e-9c91-9120337fa062 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:38:19.736233 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d83e94b3-b5cd-469e-9c91-9120337fa062 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "DELETE /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 204 len: 0 microversion: 2.1 time: 0.272125 Aug 21 01:38:19.736830 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 549/1114] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:38:19 2022] DELETE /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 0 bytes in 274 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:38:19.745399 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-68e91bd7-54d0-4c4a-92a5-914608caebe6 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:19.746121 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-68e91bd7-54d0-4c4a-92a5-914608caebe6 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:19.753377 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68e91bd7-54d0-4c4a-92a5-914608caebe6 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:19.767965 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68e91bd7-54d0-4c4a-92a5-914608caebe6 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:19.767965 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68e91bd7-54d0-4c4a-92a5-914608caebe6 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:19.767965 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68e91bd7-54d0-4c4a-92a5-914608caebe6 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:19.767965 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68e91bd7-54d0-4c4a-92a5-914608caebe6 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:19.768468 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-68e91bd7-54d0-4c4a-92a5-914608caebe6 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:19.886305 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-68e91bd7-54d0-4c4a-92a5-914608caebe6 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:19.893959 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-68e91bd7-54d0-4c4a-92a5-914608caebe6 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1504 microversion: 2.1 time: 0.152585 Aug 21 01:38:19.894517 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 566/1115] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:19 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1504 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:38:20.916236 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-23c4bb32-9ff2-43b9-aa04-59d13a5ef3fb tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:20.916929 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-23c4bb32-9ff2-43b9-aa04-59d13a5ef3fb tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:20.923389 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-23c4bb32-9ff2-43b9-aa04-59d13a5ef3fb tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:20.923865 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-23c4bb32-9ff2-43b9-aa04-59d13a5ef3fb tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:20.924327 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-23c4bb32-9ff2-43b9-aa04-59d13a5ef3fb tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:20.925014 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-23c4bb32-9ff2-43b9-aa04-59d13a5ef3fb tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:20.925459 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-23c4bb32-9ff2-43b9-aa04-59d13a5ef3fb tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:20.925882 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-23c4bb32-9ff2-43b9-aa04-59d13a5ef3fb tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:21.061561 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-23c4bb32-9ff2-43b9-aa04-59d13a5ef3fb tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:21.069546 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-23c4bb32-9ff2-43b9-aa04-59d13a5ef3fb tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1504 microversion: 2.1 time: 0.158059 Aug 21 01:38:21.070114 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 550/1116] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:20 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1504 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:22.091002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-22bc4c08-6fa9-40de-af6b-30169623d008 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:22.091654 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-22bc4c08-6fa9-40de-af6b-30169623d008 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:22.100335 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22bc4c08-6fa9-40de-af6b-30169623d008 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:22.100335 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22bc4c08-6fa9-40de-af6b-30169623d008 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:22.100335 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22bc4c08-6fa9-40de-af6b-30169623d008 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:22.100335 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22bc4c08-6fa9-40de-af6b-30169623d008 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:22.100763 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22bc4c08-6fa9-40de-af6b-30169623d008 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:22.100763 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-22bc4c08-6fa9-40de-af6b-30169623d008 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:22.207760 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-22bc4c08-6fa9-40de-af6b-30169623d008 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:22.216602 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-22bc4c08-6fa9-40de-af6b-30169623d008 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 200 len: 1462 microversion: 2.1 time: 0.129501 Aug 21 01:38:22.217189 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 567/1117] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:22 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 1462 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:22.219102 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-b55d2871-4e8b-4ef0-a7c9-eafccee12ba1 req-de4b20e6-fd1a-419d-bf2c-b1673f36893d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "146ae855-7a33-4572-ad09-b54ba7de0bcc", "name": "network-vif-deleted", "tag": "4c56ba2b-c08c-48d3-a24b-3e57d0363175"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:38:22.229410 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-b55d2871-4e8b-4ef0-a7c9-eafccee12ba1 req-de4b20e6-fd1a-419d-bf2c-b1673f36893d service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:22.237881 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-b55d2871-4e8b-4ef0-a7c9-eafccee12ba1 req-de4b20e6-fd1a-419d-bf2c-b1673f36893d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:22.237881 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-b55d2871-4e8b-4ef0-a7c9-eafccee12ba1 req-de4b20e6-fd1a-419d-bf2c-b1673f36893d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:22.244169 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-b55d2871-4e8b-4ef0-a7c9-eafccee12ba1 req-de4b20e6-fd1a-419d-bf2c-b1673f36893d service nova] Creating event network-vif-deleted:4c56ba2b-c08c-48d3-a24b-3e57d0363175 for instance 146ae855-7a33-4572-ad09-b54ba7de0bcc on hv-cinder-maste Aug 21 01:38:22.248940 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-b55d2871-4e8b-4ef0-a7c9-eafccee12ba1 req-de4b20e6-fd1a-419d-bf2c-b1673f36893d service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033478 Aug 21 01:38:22.249535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 551/1118] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:38:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:38:23.238003 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2615a081-f77a-4809-81af-63171a900026 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:23.238616 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-2615a081-f77a-4809-81af-63171a900026 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 146ae855-7a33-4572-ad09-b54ba7de0bcc] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:23.246033 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2615a081-f77a-4809-81af-63171a900026 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:23.246529 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2615a081-f77a-4809-81af-63171a900026 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:23.246966 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2615a081-f77a-4809-81af-63171a900026 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:23.247791 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2615a081-f77a-4809-81af-63171a900026 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:23.248279 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2615a081-f77a-4809-81af-63171a900026 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:23.248707 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-2615a081-f77a-4809-81af-63171a900026 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:23.263562 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-2615a081-f77a-4809-81af-63171a900026 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] HTTP exception thrown: Instance 146ae855-7a33-4572-ad09-b54ba7de0bcc could not be found. Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2615a081-f77a-4809-81af-63171a900026 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Request method failure captured: Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc HTTP/1.1 Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYx56XvqHxVgdPhMVBQtBZaB0a3jdYskHD7NiPjoCkJ9IK9JIpiwJZ6aKq4AJpaeQO4hnmsyPUxW5UcLZ1uG8XNRkGyGdtscbGrTaenS5daeSlB3TXJE737TNr5AYxc41bBsp4Vvg0Fme0oK6Qy8tKl7VFq0i3w8-hFy8ztN3PHa7UQ Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 5d895d8b54ac4d23aaca3e1065b6035f Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-VolumesActionsTest-17432435 Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:38:23.265355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:38:23.267061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/5d895d8b54ac4d23aaca3e1065b6035f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_5d895d8b54ac4d23aaca3e1065b6035f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/5d895d8b54ac4d23aaca3e1065b6035f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/5d895d8b54ac4d23aaca3e1065b6035f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:38:23.267061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-VolumesActionsTest-17432435 Aug 21 01:38:23.267061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 5d895d8b54ac4d23aaca3e1065b6035f Aug 21 01:38:23.267061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-VolumesActionsTest-17432435 Aug 21 01:38:23.267061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-VolumesActionsTest-17432435-project Aug 21 01:38:23.267061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:38:23.267061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:38:23.267061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: 08e7e7eadd5846d29c28e6e3b5c5d76f Aug 21 01:38:23.267061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-VolumesActionsTest-17432435-project Aug 21 01:38:23.267061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:38:23.267061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 146ae855-7a33-4572-ad09-b54ba7de0bcc could not be found. Aug 21 01:38:23.267061 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '146ae855-7a33-4572-ad09-b54ba7de0bcc'} Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 146ae855-7a33-4572-ad09-b54ba7de0bcc could not be found. Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:38:23.268185 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 146ae855-7a33-4572-ad09-b54ba7de0bcc could not be found. Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 146ae855-7a33-4572-ad09-b54ba7de0bcc could not be found. Aug 21 01:38:23.270463 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:38:23.272129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-2615a081-f77a-4809-81af-63171a900026 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Returning 404 to user: Instance 146ae855-7a33-4572-ad09-b54ba7de0bcc could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:38:23.272129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-2615a081-f77a-4809-81af-63171a900026 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc" status: 404 len: 111 microversion: 2.1 time: 0.032687 Aug 21 01:38:23.272129 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 568/1119] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:23 2022] GET /compute/v2.1/servers/146ae855-7a33-4572-ad09-b54ba7de0bcc => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:38:23.319406 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1826255317", "imageRef": "405f3f10-d26b-4784-a76e-85e60b2787d1", "flavorRef": "452", "networks": [{"uuid": "0e2a927d-deb9-4756-b3d9-f933a82be077"}]}} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:38:23.372895 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.network.neutron [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] validate_networks() for [('0e2a927d-deb9-4756-b3d9-f933a82be077', None, None, None, None, None)] {{(pid=121276) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:38:23.725953 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Flavor limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:38:23.726376 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Image limits 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:38:23.727145 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Flavor pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:38:23.727374 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Image pref 0:0:0 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:38:23.727633 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.virt.hardware [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121276) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:38:23.736503 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:23.736983 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:23.737420 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:23.738177 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:23.738634 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:23.739061 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:23.756635 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Getting quotas for project 5d895d8b54ac4d23aaca3e1065b6035f. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:38:23.760599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.quota [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Getting quotas for user 08e7e7eadd5846d29c28e6e3b5c5d76f and project 5d895d8b54ac4d23aaca3e1065b6035f. Resources: {'instances', 'ram', 'cores'} {{(pid=121276) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:38:23.773128 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Going to run 1 instances... {{(pid=121276) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:38:23.775766 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='405f3f10-d26b-4784-a76e-85e60b2787d1',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121276) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:38:23.943578 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-c8ffca9f-505c-411e-976a-3e9707026625 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.629259 Aug 21 01:38:23.944263 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 552/1120] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:38:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 631 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:38:23.955081 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-567661f7-1fee-422e-ad50-76aab9714c9a tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:23.955700 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-567661f7-1fee-422e-ad50-76aab9714c9a tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:24.012446 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-567661f7-1fee-422e-ad50-76aab9714c9a tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:24.019967 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-567661f7-1fee-422e-ad50-76aab9714c9a tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1214 microversion: 2.1 time: 0.068801 Aug 21 01:38:24.020568 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 569/1121] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:23 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1214 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:25.041298 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-88c47f2c-3ac2-464e-a2ec-42d74b9c1461 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:25.041970 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-88c47f2c-3ac2-464e-a2ec-42d74b9c1461 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:25.048851 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-88c47f2c-3ac2-464e-a2ec-42d74b9c1461 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:25.049289 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-88c47f2c-3ac2-464e-a2ec-42d74b9c1461 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:25.049712 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-88c47f2c-3ac2-464e-a2ec-42d74b9c1461 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:25.050406 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-88c47f2c-3ac2-464e-a2ec-42d74b9c1461 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:25.050861 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-88c47f2c-3ac2-464e-a2ec-42d74b9c1461 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:25.051291 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-88c47f2c-3ac2-464e-a2ec-42d74b9c1461 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:25.113683 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-88c47f2c-3ac2-464e-a2ec-42d74b9c1461 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:25.121646 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-88c47f2c-3ac2-464e-a2ec-42d74b9c1461 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1218 microversion: 2.1 time: 0.084225 Aug 21 01:38:25.122223 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 553/1122] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:25 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1218 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:26.144935 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8e492f62-9e28-497c-98c0-ec23ff59962e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:26.146186 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8e492f62-9e28-497c-98c0-ec23ff59962e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:26.154510 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e492f62-9e28-497c-98c0-ec23ff59962e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:26.155002 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e492f62-9e28-497c-98c0-ec23ff59962e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:26.155436 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e492f62-9e28-497c-98c0-ec23ff59962e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:26.156176 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e492f62-9e28-497c-98c0-ec23ff59962e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:26.156702 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e492f62-9e28-497c-98c0-ec23ff59962e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:26.157327 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8e492f62-9e28-497c-98c0-ec23ff59962e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:26.256160 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8e492f62-9e28-497c-98c0-ec23ff59962e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:26.272383 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8e492f62-9e28-497c-98c0-ec23ff59962e tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1274 microversion: 2.1 time: 0.125474 Aug 21 01:38:26.273761 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 570/1123] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:26 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1274 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:27.294091 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-09b60150-f296-481d-bc3b-313ff8c228d7 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:27.294754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-09b60150-f296-481d-bc3b-313ff8c228d7 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:27.301180 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-09b60150-f296-481d-bc3b-313ff8c228d7 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:27.307947 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-09b60150-f296-481d-bc3b-313ff8c228d7 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:27.307947 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-09b60150-f296-481d-bc3b-313ff8c228d7 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:27.307947 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-09b60150-f296-481d-bc3b-313ff8c228d7 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:27.307947 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-09b60150-f296-481d-bc3b-313ff8c228d7 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:27.308417 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-09b60150-f296-481d-bc3b-313ff8c228d7 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:27.454607 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-09b60150-f296-481d-bc3b-313ff8c228d7 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:27.462577 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-09b60150-f296-481d-bc3b-313ff8c228d7 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1314 microversion: 2.1 time: 0.172509 Aug 21 01:38:27.463138 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 554/1124] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:27 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1314 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:27.840571 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-80b3c14f-59e6-440d-a1b7-e4cb36cc8054 req-fe7334ea-3854-436e-b5b7-ccfcf8c57168 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "30167b0f-2ddb-4339-b09f-bc7911838a9f", "tag": "a0add147-c561-43dc-9b8c-c1ad7518f442"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:38:27.851765 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-80b3c14f-59e6-440d-a1b7-e4cb36cc8054 req-fe7334ea-3854-436e-b5b7-ccfcf8c57168 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:27.852552 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-80b3c14f-59e6-440d-a1b7-e4cb36cc8054 req-fe7334ea-3854-436e-b5b7-ccfcf8c57168 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:27.853007 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-80b3c14f-59e6-440d-a1b7-e4cb36cc8054 req-fe7334ea-3854-436e-b5b7-ccfcf8c57168 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:27.864692 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-80b3c14f-59e6-440d-a1b7-e4cb36cc8054 req-fe7334ea-3854-436e-b5b7-ccfcf8c57168 service nova] Creating event network-changed:a0add147-c561-43dc-9b8c-c1ad7518f442 for instance 30167b0f-2ddb-4339-b09f-bc7911838a9f on hv-cinder-maste Aug 21 01:38:27.869537 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-80b3c14f-59e6-440d-a1b7-e4cb36cc8054 req-fe7334ea-3854-436e-b5b7-ccfcf8c57168 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035044 Aug 21 01:38:27.870134 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 571/1125] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:38:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:38:28.482717 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-8e265f71-7e3b-49c1-b578-917e1ab088f0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:28.483340 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-8e265f71-7e3b-49c1-b578-917e1ab088f0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:28.489908 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e265f71-7e3b-49c1-b578-917e1ab088f0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:28.490920 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e265f71-7e3b-49c1-b578-917e1ab088f0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:28.491373 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e265f71-7e3b-49c1-b578-917e1ab088f0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:28.492077 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e265f71-7e3b-49c1-b578-917e1ab088f0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:28.492558 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e265f71-7e3b-49c1-b578-917e1ab088f0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:28.492990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-8e265f71-7e3b-49c1-b578-917e1ab088f0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:28.691759 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-8e265f71-7e3b-49c1-b578-917e1ab088f0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:28.700522 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-8e265f71-7e3b-49c1-b578-917e1ab088f0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1314 microversion: 2.1 time: 0.221550 Aug 21 01:38:28.701104 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 555/1126] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:28 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1314 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:29.723970 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9176b091-7f8c-4905-9f79-ce5cd186186d tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:29.724696 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9176b091-7f8c-4905-9f79-ce5cd186186d tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:29.730949 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9176b091-7f8c-4905-9f79-ce5cd186186d tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:29.731435 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9176b091-7f8c-4905-9f79-ce5cd186186d tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:29.731875 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9176b091-7f8c-4905-9f79-ce5cd186186d tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:29.732732 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9176b091-7f8c-4905-9f79-ce5cd186186d tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:29.733193 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9176b091-7f8c-4905-9f79-ce5cd186186d tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:29.733621 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9176b091-7f8c-4905-9f79-ce5cd186186d tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:29.863851 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9176b091-7f8c-4905-9f79-ce5cd186186d tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:29.871572 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9176b091-7f8c-4905-9f79-ce5cd186186d tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1314 microversion: 2.1 time: 0.152356 Aug 21 01:38:29.872151 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 572/1127] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:29 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:30.892544 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-504f2cfe-bba3-42e3-9496-5e448b6920e9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:30.893407 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-504f2cfe-bba3-42e3-9496-5e448b6920e9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:30.900048 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-504f2cfe-bba3-42e3-9496-5e448b6920e9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:30.900571 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-504f2cfe-bba3-42e3-9496-5e448b6920e9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:30.900995 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-504f2cfe-bba3-42e3-9496-5e448b6920e9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:30.901693 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-504f2cfe-bba3-42e3-9496-5e448b6920e9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:30.902141 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-504f2cfe-bba3-42e3-9496-5e448b6920e9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:30.902564 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-504f2cfe-bba3-42e3-9496-5e448b6920e9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:31.034582 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-504f2cfe-bba3-42e3-9496-5e448b6920e9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:31.042376 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-504f2cfe-bba3-42e3-9496-5e448b6920e9 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1314 microversion: 2.1 time: 0.153754 Aug 21 01:38:31.042936 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 556/1128] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:30 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1314 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:32.063367 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5addb637-0946-410e-9005-551af4c5c194 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:32.063996 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5addb637-0946-410e-9005-551af4c5c194 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:32.070965 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5addb637-0946-410e-9005-551af4c5c194 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:32.071452 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5addb637-0946-410e-9005-551af4c5c194 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:32.071890 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5addb637-0946-410e-9005-551af4c5c194 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:32.072614 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5addb637-0946-410e-9005-551af4c5c194 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:32.073069 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5addb637-0946-410e-9005-551af4c5c194 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:32.073495 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5addb637-0946-410e-9005-551af4c5c194 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:32.204888 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5addb637-0946-410e-9005-551af4c5c194 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:32.213590 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5addb637-0946-410e-9005-551af4c5c194 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1314 microversion: 2.1 time: 0.154223 Aug 21 01:38:32.214155 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 573/1129] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:32 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1314 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:33.234212 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-5c9eb8d8-1125-4d3f-9c26-0a43a8069ab0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:33.234855 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-5c9eb8d8-1125-4d3f-9c26-0a43a8069ab0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:33.242209 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5c9eb8d8-1125-4d3f-9c26-0a43a8069ab0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:33.242702 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5c9eb8d8-1125-4d3f-9c26-0a43a8069ab0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:33.243149 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5c9eb8d8-1125-4d3f-9c26-0a43a8069ab0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:33.243850 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5c9eb8d8-1125-4d3f-9c26-0a43a8069ab0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:33.244349 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5c9eb8d8-1125-4d3f-9c26-0a43a8069ab0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:33.244779 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-5c9eb8d8-1125-4d3f-9c26-0a43a8069ab0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:33.380878 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-5c9eb8d8-1125-4d3f-9c26-0a43a8069ab0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:33.388734 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-5c9eb8d8-1125-4d3f-9c26-0a43a8069ab0 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1314 microversion: 2.1 time: 0.158579 Aug 21 01:38:33.389302 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 557/1130] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:33 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:33.707241 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-0330a67f-fcac-4d06-bd32-e9eb7bab524f req-90cd7352-270f-438a-9f09-7dbdeadf3cd5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "30167b0f-2ddb-4339-b09f-bc7911838a9f", "name": "network-vif-plugged", "status": "completed", "tag": "a0add147-c561-43dc-9b8c-c1ad7518f442"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:38:33.717629 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-0330a67f-fcac-4d06-bd32-e9eb7bab524f req-90cd7352-270f-438a-9f09-7dbdeadf3cd5 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:33.718012 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-0330a67f-fcac-4d06-bd32-e9eb7bab524f req-90cd7352-270f-438a-9f09-7dbdeadf3cd5 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:33.718436 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-0330a67f-fcac-4d06-bd32-e9eb7bab524f req-90cd7352-270f-438a-9f09-7dbdeadf3cd5 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:33.731925 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-0330a67f-fcac-4d06-bd32-e9eb7bab524f req-90cd7352-270f-438a-9f09-7dbdeadf3cd5 service nova] Creating event network-vif-plugged:a0add147-c561-43dc-9b8c-c1ad7518f442 for instance 30167b0f-2ddb-4339-b09f-bc7911838a9f on hv-cinder-maste Aug 21 01:38:33.736644 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-0330a67f-fcac-4d06-bd32-e9eb7bab524f req-90cd7352-270f-438a-9f09-7dbdeadf3cd5 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033272 Aug 21 01:38:33.737428 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 574/1131] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:38:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:38:34.409241 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-2b3b72ea-480d-4511-b3df-06c5ddf81e67 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:34.409853 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-2b3b72ea-480d-4511-b3df-06c5ddf81e67 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:34.411771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:34.412489 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=6 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:34.413583 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=62,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:34.418290 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2b3b72ea-480d-4511-b3df-06c5ddf81e67 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:34.418774 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2b3b72ea-480d-4511-b3df-06c5ddf81e67 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:34.419218 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2b3b72ea-480d-4511-b3df-06c5ddf81e67 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:34.419948 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2b3b72ea-480d-4511-b3df-06c5ddf81e67 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:34.420427 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2b3b72ea-480d-4511-b3df-06c5ddf81e67 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:34.423742 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-2b3b72ea-480d-4511-b3df-06c5ddf81e67 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:34.427589 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:34.555975 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-2b3b72ea-480d-4511-b3df-06c5ddf81e67 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:34.563632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-2b3b72ea-480d-4511-b3df-06c5ddf81e67 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1314 microversion: 2.1 time: 0.158237 Aug 21 01:38:34.564388 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 558/1132] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:34 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:35.580941 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:35.586257 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-b4c70f4f-8451-43a9-8abc-e756950738fc tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:35.586970 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-b4c70f4f-8451-43a9-8abc-e756950738fc tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:35.588742 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=39 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:35.593569 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b4c70f4f-8451-43a9-8abc-e756950738fc tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:35.594052 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b4c70f4f-8451-43a9-8abc-e756950738fc tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:35.594489 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b4c70f4f-8451-43a9-8abc-e756950738fc tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:35.595254 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b4c70f4f-8451-43a9-8abc-e756950738fc tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:35.595701 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b4c70f4f-8451-43a9-8abc-e756950738fc tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:35.596146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-b4c70f4f-8451-43a9-8abc-e756950738fc tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:35.730725 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-b4c70f4f-8451-43a9-8abc-e756950738fc tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:35.738331 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-b4c70f4f-8451-43a9-8abc-e756950738fc tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1498 microversion: 2.1 time: 0.158347 Aug 21 01:38:35.738928 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 575/1133] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:35 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1498 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:36.200688 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:38:36.201028 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Sun Aug 21 01:38:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:38:36.715997 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e0d7969a-6ddf-49f2-9e3c-ba30c794d6d1 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:36.716581 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e0d7969a-6ddf-49f2-9e3c-ba30c794d6d1 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:36.724024 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e0d7969a-6ddf-49f2-9e3c-ba30c794d6d1 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:36.724535 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e0d7969a-6ddf-49f2-9e3c-ba30c794d6d1 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:36.725467 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e0d7969a-6ddf-49f2-9e3c-ba30c794d6d1 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:36.726157 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e0d7969a-6ddf-49f2-9e3c-ba30c794d6d1 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:36.726615 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e0d7969a-6ddf-49f2-9e3c-ba30c794d6d1 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:36.727039 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e0d7969a-6ddf-49f2-9e3c-ba30c794d6d1 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:36.751906 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e0d7969a-6ddf-49f2-9e3c-ba30c794d6d1 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:38:37.015483 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e0d7969a-6ddf-49f2-9e3c-ba30c794d6d1 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "DELETE /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 204 len: 0 microversion: 2.1 time: 0.303014 Aug 21 01:38:37.016066 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 559/1134] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:38:36 2022] DELETE /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:38:37.024940 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-bbab7c5a-8626-4409-8a0d-9e2652478729 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:37.025553 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-bbab7c5a-8626-4409-8a0d-9e2652478729 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:37.032243 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bbab7c5a-8626-4409-8a0d-9e2652478729 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:37.032728 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bbab7c5a-8626-4409-8a0d-9e2652478729 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:37.033166 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bbab7c5a-8626-4409-8a0d-9e2652478729 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:37.033856 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bbab7c5a-8626-4409-8a0d-9e2652478729 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:37.034312 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bbab7c5a-8626-4409-8a0d-9e2652478729 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:37.034734 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-bbab7c5a-8626-4409-8a0d-9e2652478729 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:37.038221 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=81 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:38:37.164134 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-bbab7c5a-8626-4409-8a0d-9e2652478729 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:37.172091 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-bbab7c5a-8626-4409-8a0d-9e2652478729 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1504 microversion: 2.1 time: 0.151703 Aug 21 01:38:37.175186 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 576/1135] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:37 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1504 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:38.192825 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f4c83086-19fb-43f4-82c9-d68afb806850 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:38.193677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f4c83086-19fb-43f4-82c9-d68afb806850 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:38.200193 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c83086-19fb-43f4-82c9-d68afb806850 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:38.200787 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c83086-19fb-43f4-82c9-d68afb806850 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:38.201232 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c83086-19fb-43f4-82c9-d68afb806850 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:38.201926 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c83086-19fb-43f4-82c9-d68afb806850 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:38.202377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c83086-19fb-43f4-82c9-d68afb806850 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:38.202802 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c83086-19fb-43f4-82c9-d68afb806850 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:38.335984 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f4c83086-19fb-43f4-82c9-d68afb806850 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:38.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f4c83086-19fb-43f4-82c9-d68afb806850 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1504 microversion: 2.1 time: 0.155491 Aug 21 01:38:38.344879 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 560/1136] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:38 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1504 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:39.177362 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-cbe17509-9ad6-4348-8f6f-91fd8fe49286 req-0ae0b2b9-a576-4898-94ab-b420b16a8151 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "30167b0f-2ddb-4339-b09f-bc7911838a9f", "name": "network-vif-deleted", "tag": "a0add147-c561-43dc-9b8c-c1ad7518f442"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:38:39.187442 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-cbe17509-9ad6-4348-8f6f-91fd8fe49286 req-0ae0b2b9-a576-4898-94ab-b420b16a8151 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:39.187917 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-cbe17509-9ad6-4348-8f6f-91fd8fe49286 req-0ae0b2b9-a576-4898-94ab-b420b16a8151 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:39.188375 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-cbe17509-9ad6-4348-8f6f-91fd8fe49286 req-0ae0b2b9-a576-4898-94ab-b420b16a8151 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:39.199914 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-cbe17509-9ad6-4348-8f6f-91fd8fe49286 req-0ae0b2b9-a576-4898-94ab-b420b16a8151 service nova] Creating event network-vif-deleted:a0add147-c561-43dc-9b8c-c1ad7518f442 for instance 30167b0f-2ddb-4339-b09f-bc7911838a9f on hv-cinder-maste Aug 21 01:38:39.204976 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-cbe17509-9ad6-4348-8f6f-91fd8fe49286 req-0ae0b2b9-a576-4898-94ab-b420b16a8151 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030979 Aug 21 01:38:39.205252 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 577/1137] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:38:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:38:39.366287 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-919da571-ac34-4ace-8bb5-81c581c86f34 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:39.366906 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-919da571-ac34-4ace-8bb5-81c581c86f34 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:39.373574 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-919da571-ac34-4ace-8bb5-81c581c86f34 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:39.374055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-919da571-ac34-4ace-8bb5-81c581c86f34 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:39.374490 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-919da571-ac34-4ace-8bb5-81c581c86f34 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:39.375181 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-919da571-ac34-4ace-8bb5-81c581c86f34 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:39.375636 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-919da571-ac34-4ace-8bb5-81c581c86f34 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:39.376069 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-919da571-ac34-4ace-8bb5-81c581c86f34 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:39.439438 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-919da571-ac34-4ace-8bb5-81c581c86f34 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:39.447704 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-919da571-ac34-4ace-8bb5-81c581c86f34 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1299 microversion: 2.1 time: 0.086082 Aug 21 01:38:39.448293 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 561/1138] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:39 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1299 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:40.468883 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:40.469677 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:40.476681 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:40.477165 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:40.477601 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:40.478294 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:40.478744 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:40.479166 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:40.511354 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:40.511748 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:40.512234 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:40.512681 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.objects.instance [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lazy-loading 'fault' on Instance uuid 30167b0f-2ddb-4339-b09f-bc7911838a9f {{(pid=121277) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:38:40.557593 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08e7e7eadd5846d29c28e6e3b5c5d76f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d895d8b54ac4d23aaca3e1065b6035f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:38:40.565365 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-5d070d88-5f65-4456-87f0-39c7dc0dc41c tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 200 len: 1304 microversion: 2.1 time: 0.100597 Aug 21 01:38:40.565940 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 578/1139] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:40 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 1304 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:38:41.587165 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e112be5c-ca45-4582-9f27-93655455a286 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:38:41.587780 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-e112be5c-ca45-4582-9f27-93655455a286 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] [instance: 30167b0f-2ddb-4339-b09f-bc7911838a9f] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:38:41.594429 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e112be5c-ca45-4582-9f27-93655455a286 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:41.594914 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e112be5c-ca45-4582-9f27-93655455a286 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:41.595353 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e112be5c-ca45-4582-9f27-93655455a286 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:41.596046 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e112be5c-ca45-4582-9f27-93655455a286 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:38:41.596522 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e112be5c-ca45-4582-9f27-93655455a286 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:38:41.596951 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-e112be5c-ca45-4582-9f27-93655455a286 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:38:41.610727 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-e112be5c-ca45-4582-9f27-93655455a286 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] HTTP exception thrown: Instance 30167b0f-2ddb-4339-b09f-bc7911838a9f could not be found. Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e112be5c-ca45-4582-9f27-93655455a286 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Request method failure captured: Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f HTTP/1.1 Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYx56XvqHxVgdPhMVBQtBZaB0a3jdYskHD7NiPjoCkJ9IK9JIpiwJZ6aKq4AJpaeQO4hnmsyPUxW5UcLZ1uG8XNRkGyGdtscbGrTaenS5daeSlB3TXJE737TNr5AYxc41bBsp4Vvg0Fme0oK6Qy8tKl7VFq0i3w8-hFy8ztN3PHa7UQ Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 5d895d8b54ac4d23aaca3e1065b6035f Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-VolumesActionsTest-17432435 Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:38:41.612440 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:38:41.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/5d895d8b54ac4d23aaca3e1065b6035f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_5d895d8b54ac4d23aaca3e1065b6035f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/5d895d8b54ac4d23aaca3e1065b6035f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/5d895d8b54ac4d23aaca3e1065b6035f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:38:41.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-VolumesActionsTest-17432435 Aug 21 01:38:41.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 5d895d8b54ac4d23aaca3e1065b6035f Aug 21 01:38:41.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-VolumesActionsTest-17432435 Aug 21 01:38:41.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-VolumesActionsTest-17432435-project Aug 21 01:38:41.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:38:41.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:38:41.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: 08e7e7eadd5846d29c28e6e3b5c5d76f Aug 21 01:38:41.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-VolumesActionsTest-17432435-project Aug 21 01:38:41.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:38:41.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 30167b0f-2ddb-4339-b09f-bc7911838a9f could not be found. Aug 21 01:38:41.614182 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '30167b0f-2ddb-4339-b09f-bc7911838a9f'} Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 30167b0f-2ddb-4339-b09f-bc7911838a9f could not be found. Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:38:41.615267 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 30167b0f-2ddb-4339-b09f-bc7911838a9f could not be found. Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 30167b0f-2ddb-4339-b09f-bc7911838a9f could not be found. Aug 21 01:38:41.617529 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:38:41.619032 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-e112be5c-ca45-4582-9f27-93655455a286 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] Returning 404 to user: Instance 30167b0f-2ddb-4339-b09f-bc7911838a9f could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:38:41.619032 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-e112be5c-ca45-4582-9f27-93655455a286 tempest-VolumesActionsTest-17432435 tempest-VolumesActionsTest-17432435-project] 10.222.0.47 "GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f" status: 404 len: 111 microversion: 2.1 time: 0.030982 Aug 21 01:38:41.619032 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 562/1140] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:38:41 2022] GET /compute/v2.1/servers/30167b0f-2ddb-4339-b09f-bc7911838a9f => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:39:19.951169 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-146680105", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1465137419", "security_groups": [{"name": "tempest-secgroup-smoke-1796196116"}], "block_device_mapping_v2": [{"uuid": "8b2c3f2d-3cd5-4606-8da9-5ddab02d5e80", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "e4ca6005-16bf-4430-905e-1ccbd3a7f5d3"}]}} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:39:19.961146 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=9 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:39:19.961957 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=15 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:39:20.068333 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.network.neutron [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] validate_networks() for [('e4ca6005-16bf-4430-905e-1ccbd3a7f5d3', None, None, None, None, None)] {{(pid=121277) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 21 01:39:20.802408 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Flavor limits 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 21 01:39:20.802832 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Image limits 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 21 01:39:20.803308 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Flavor pref 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 21 01:39:20.803718 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Image pref 0:0:0 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 21 01:39:20.804175 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.virt.hardware [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121277) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 21 01:39:20.815675 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:20.816208 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:20.816649 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:20.817373 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:20.817830 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:20.818345 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:20.835617 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Getting quotas for project 4124766f4f2843229059dbbdf7a2de74. Resources: {'instances', 'ram', 'cores'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 21 01:39:20.839563 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.quota [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Getting quotas for user fef6461d7ca14cab829412ce3b53852a and project 4124766f4f2843229059dbbdf7a2de74. Resources: {'instances', 'ram', 'cores'} {{(pid=121277) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 21 01:39:20.896538 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Going to run 1 instances... {{(pid=121277) _provision_instances /opt/stack/nova/nova/compute/api.py:1390}} Aug 21 01:39:20.898826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='8b2c3f2d-3cd5-4606-8da9-5ddab02d5e80',volume_size=None,volume_type=None)] {{(pid=121277) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1851}} Aug 21 01:39:21.228429 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-83fed6de-1390-40c7-99fe-9524ee63422b tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.281746 Aug 21 01:39:21.229908 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 579/1141] 10.222.0.47 () {60 vars in 1147 bytes} [Sun Aug 21 01:39:19 2022] POST /compute/v2.1/servers => generated 406 bytes in 1283 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 21 01:39:21.240602 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-dd9b7f3e-6959-4d1d-9b57-65655d04772f tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:21.241221 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-dd9b7f3e-6959-4d1d-9b57-65655d04772f tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:21.243526 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=14,nova_api:UPDATE=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:39:21.244331 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=33,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:39:21.297347 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-dd9b7f3e-6959-4d1d-9b57-65655d04772f tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:21.304470 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-dd9b7f3e-6959-4d1d-9b57-65655d04772f tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1094 microversion: 2.1 time: 0.067922 Aug 21 01:39:21.305034 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 563/1142] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:21 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1094 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:39:22.326243 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-29236857-8756-422f-8b73-e6e97db261cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:22.326880 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-29236857-8756-422f-8b73-e6e97db261cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:22.332995 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-29236857-8756-422f-8b73-e6e97db261cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:22.336290 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-29236857-8756-422f-8b73-e6e97db261cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:22.336290 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-29236857-8756-422f-8b73-e6e97db261cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:22.336290 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-29236857-8756-422f-8b73-e6e97db261cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:22.336290 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-29236857-8756-422f-8b73-e6e97db261cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:22.336704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-29236857-8756-422f-8b73-e6e97db261cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:22.397959 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-29236857-8756-422f-8b73-e6e97db261cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:22.406501 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-29236857-8756-422f-8b73-e6e97db261cb tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1144 microversion: 2.1 time: 0.085071 Aug 21 01:39:22.406690 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 580/1143] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:22 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1144 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:23.427433 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-4ced4c5d-95b7-48f0-a5bc-d2304045924d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:23.428055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-4ced4c5d-95b7-48f0-a5bc-d2304045924d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:23.437139 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ced4c5d-95b7-48f0-a5bc-d2304045924d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:23.437632 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ced4c5d-95b7-48f0-a5bc-d2304045924d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:23.438071 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ced4c5d-95b7-48f0-a5bc-d2304045924d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:23.438779 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ced4c5d-95b7-48f0-a5bc-d2304045924d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:23.439233 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ced4c5d-95b7-48f0-a5bc-d2304045924d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:23.439661 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-4ced4c5d-95b7-48f0-a5bc-d2304045924d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:23.517854 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-4ced4c5d-95b7-48f0-a5bc-d2304045924d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:23.525892 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-4ced4c5d-95b7-48f0-a5bc-d2304045924d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1200 microversion: 2.1 time: 0.102456 Aug 21 01:39:23.530246 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 564/1144] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:23 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1200 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:24.547225 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-81f039bc-956b-431e-ad89-1d995f93a639 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:24.547870 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-81f039bc-956b-431e-ad89-1d995f93a639 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:24.554303 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-81f039bc-956b-431e-ad89-1d995f93a639 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:24.554792 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-81f039bc-956b-431e-ad89-1d995f93a639 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:24.555219 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-81f039bc-956b-431e-ad89-1d995f93a639 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:24.555912 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-81f039bc-956b-431e-ad89-1d995f93a639 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:24.556921 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-81f039bc-956b-431e-ad89-1d995f93a639 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:24.557359 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-81f039bc-956b-431e-ad89-1d995f93a639 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:24.759493 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-81f039bc-956b-431e-ad89-1d995f93a639 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:24.767415 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-81f039bc-956b-431e-ad89-1d995f93a639 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1278 microversion: 2.1 time: 0.224212 Aug 21 01:39:24.767974 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 581/1145] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:24 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1278 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:25.049021 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-10dfc024-bb6e-46d4-800e-11d00f635fc9 req-4c79ac49-eacd-4f22-9001-4d3526b06fee service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f76f93df-1be7-4260-bcb6-95563c549933", "tag": "e5006064-c051-4d6e-b083-787006f20c1f"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:39:25.061834 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-10dfc024-bb6e-46d4-800e-11d00f635fc9 req-4c79ac49-eacd-4f22-9001-4d3526b06fee service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:25.062318 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-10dfc024-bb6e-46d4-800e-11d00f635fc9 req-4c79ac49-eacd-4f22-9001-4d3526b06fee service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:25.062752 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-10dfc024-bb6e-46d4-800e-11d00f635fc9 req-4c79ac49-eacd-4f22-9001-4d3526b06fee service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:25.074868 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-10dfc024-bb6e-46d4-800e-11d00f635fc9 req-4c79ac49-eacd-4f22-9001-4d3526b06fee service nova] Creating event network-changed:e5006064-c051-4d6e-b083-787006f20c1f for instance f76f93df-1be7-4260-bcb6-95563c549933 on hv-cinder-maste Aug 21 01:39:25.080305 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-10dfc024-bb6e-46d4-800e-11d00f635fc9 req-4c79ac49-eacd-4f22-9001-4d3526b06fee service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034923 Aug 21 01:39:25.080906 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 565/1146] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:39:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 21 01:39:25.788299 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-85edde8f-3933-43bb-8e31-f8b7a86c4115 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:25.788944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-85edde8f-3933-43bb-8e31-f8b7a86c4115 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:25.795269 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-85edde8f-3933-43bb-8e31-f8b7a86c4115 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:25.795759 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-85edde8f-3933-43bb-8e31-f8b7a86c4115 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:25.796224 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-85edde8f-3933-43bb-8e31-f8b7a86c4115 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:25.796931 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-85edde8f-3933-43bb-8e31-f8b7a86c4115 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:25.797379 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-85edde8f-3933-43bb-8e31-f8b7a86c4115 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:25.797805 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-85edde8f-3933-43bb-8e31-f8b7a86c4115 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:25.952078 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-85edde8f-3933-43bb-8e31-f8b7a86c4115 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:25.960157 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-85edde8f-3933-43bb-8e31-f8b7a86c4115 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1266 microversion: 2.1 time: 0.175800 Aug 21 01:39:25.961912 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 582/1147] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:25 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1266 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:26.981571 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-84ba469c-8708-40eb-9104-31436adfca97 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:26.982258 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-84ba469c-8708-40eb-9104-31436adfca97 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:26.988856 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-84ba469c-8708-40eb-9104-31436adfca97 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:26.989338 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-84ba469c-8708-40eb-9104-31436adfca97 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:26.989772 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-84ba469c-8708-40eb-9104-31436adfca97 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:26.990457 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-84ba469c-8708-40eb-9104-31436adfca97 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:26.990908 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-84ba469c-8708-40eb-9104-31436adfca97 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:26.991329 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-84ba469c-8708-40eb-9104-31436adfca97 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:27.129884 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-84ba469c-8708-40eb-9104-31436adfca97 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:27.138931 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-84ba469c-8708-40eb-9104-31436adfca97 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1266 microversion: 2.1 time: 0.161489 Aug 21 01:39:27.139515 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 566/1148] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:26 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:28.159704 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-9db96084-7f56-42ce-b529-e7d87a5fc523 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:28.160422 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-9db96084-7f56-42ce-b529-e7d87a5fc523 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:28.167448 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9db96084-7f56-42ce-b529-e7d87a5fc523 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:28.167932 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9db96084-7f56-42ce-b529-e7d87a5fc523 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:28.168400 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9db96084-7f56-42ce-b529-e7d87a5fc523 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:28.169099 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9db96084-7f56-42ce-b529-e7d87a5fc523 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:28.169557 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9db96084-7f56-42ce-b529-e7d87a5fc523 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:28.169979 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-9db96084-7f56-42ce-b529-e7d87a5fc523 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:28.306118 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-9db96084-7f56-42ce-b529-e7d87a5fc523 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:28.314298 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-9db96084-7f56-42ce-b529-e7d87a5fc523 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1266 microversion: 2.1 time: 0.158395 Aug 21 01:39:28.315052 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 583/1149] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:28 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1266 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:29.335022 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ce00792c-1895-4c7b-b2e9-74fbdcbfd2e4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:29.335689 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ce00792c-1895-4c7b-b2e9-74fbdcbfd2e4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:29.342317 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce00792c-1895-4c7b-b2e9-74fbdcbfd2e4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:29.342806 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce00792c-1895-4c7b-b2e9-74fbdcbfd2e4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:29.343250 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce00792c-1895-4c7b-b2e9-74fbdcbfd2e4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:29.343961 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce00792c-1895-4c7b-b2e9-74fbdcbfd2e4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:29.344445 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce00792c-1895-4c7b-b2e9-74fbdcbfd2e4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:29.344874 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce00792c-1895-4c7b-b2e9-74fbdcbfd2e4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:29.473125 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ce00792c-1895-4c7b-b2e9-74fbdcbfd2e4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:29.481272 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ce00792c-1895-4c7b-b2e9-74fbdcbfd2e4 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1266 microversion: 2.1 time: 0.150368 Aug 21 01:39:29.481829 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 567/1150] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:29 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:30.500843 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-56162594-8af1-4ecf-80a0-28d29c3612ca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:30.501382 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-56162594-8af1-4ecf-80a0-28d29c3612ca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:30.507728 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56162594-8af1-4ecf-80a0-28d29c3612ca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:30.508243 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56162594-8af1-4ecf-80a0-28d29c3612ca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:30.508687 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56162594-8af1-4ecf-80a0-28d29c3612ca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:30.509981 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56162594-8af1-4ecf-80a0-28d29c3612ca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:30.510439 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56162594-8af1-4ecf-80a0-28d29c3612ca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:30.510872 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-56162594-8af1-4ecf-80a0-28d29c3612ca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:30.642288 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-56162594-8af1-4ecf-80a0-28d29c3612ca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:30.651961 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-56162594-8af1-4ecf-80a0-28d29c3612ca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1266 microversion: 2.1 time: 0.155070 Aug 21 01:39:30.652551 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 584/1151] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:30 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1266 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:31.677677 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell0:SELECT=2 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:39:31.791355 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a5e8e17a-4eaf-4188-b8f1-bdceaffe4ced tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:31.791971 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-a5e8e17a-4eaf-4188-b8f1-bdceaffe4ced tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:31.798111 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5e8e17a-4eaf-4188-b8f1-bdceaffe4ced tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:31.798591 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5e8e17a-4eaf-4188-b8f1-bdceaffe4ced tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:31.799028 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5e8e17a-4eaf-4188-b8f1-bdceaffe4ced tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:31.799763 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5e8e17a-4eaf-4188-b8f1-bdceaffe4ced tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:31.800246 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5e8e17a-4eaf-4188-b8f1-bdceaffe4ced tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:31.805188 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-a5e8e17a-4eaf-4188-b8f1-bdceaffe4ced tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:31.929570 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-a5e8e17a-4eaf-4188-b8f1-bdceaffe4ced tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:31.938868 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a5e8e17a-4eaf-4188-b8f1-bdceaffe4ced tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1266 microversion: 2.1 time: 0.266374 Aug 21 01:39:31.939427 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 568/1152] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:31 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1266 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:32.960363 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-faa5c30d-62bb-441a-ab7e-0612049b330c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:32.960985 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-faa5c30d-62bb-441a-ab7e-0612049b330c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:32.963724 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=2 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:39:32.964540 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell0:SELECT=6 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:39:32.970067 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-faa5c30d-62bb-441a-ab7e-0612049b330c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:32.970561 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-faa5c30d-62bb-441a-ab7e-0612049b330c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:32.971035 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-faa5c30d-62bb-441a-ab7e-0612049b330c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:32.971705 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-faa5c30d-62bb-441a-ab7e-0612049b330c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:32.972189 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-faa5c30d-62bb-441a-ab7e-0612049b330c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:32.976805 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-faa5c30d-62bb-441a-ab7e-0612049b330c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:33.104948 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-faa5c30d-62bb-441a-ab7e-0612049b330c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:33.112741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-faa5c30d-62bb-441a-ab7e-0612049b330c tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1266 microversion: 2.1 time: 0.156489 Aug 21 01:39:33.115546 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 585/1153] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:32 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:34.132754 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-db26b3c0-6a9c-4887-975f-1b18e380bf7a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:34.133371 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-db26b3c0-6a9c-4887-975f-1b18e380bf7a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:34.139961 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db26b3c0-6a9c-4887-975f-1b18e380bf7a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:34.140482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db26b3c0-6a9c-4887-975f-1b18e380bf7a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:34.140984 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db26b3c0-6a9c-4887-975f-1b18e380bf7a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:34.141685 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db26b3c0-6a9c-4887-975f-1b18e380bf7a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:34.142138 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db26b3c0-6a9c-4887-975f-1b18e380bf7a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:34.142565 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-db26b3c0-6a9c-4887-975f-1b18e380bf7a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:34.271711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-db26b3c0-6a9c-4887-975f-1b18e380bf7a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:34.280862 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-db26b3c0-6a9c-4887-975f-1b18e380bf7a tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1266 microversion: 2.1 time: 0.151787 Aug 21 01:39:34.281563 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 569/1154] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:34 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1266 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:35.302133 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-1037fb34-a43a-4a7e-b838-595832697545 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:35.302729 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-1037fb34-a43a-4a7e-b838-595832697545 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:35.310053 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1037fb34-a43a-4a7e-b838-595832697545 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:35.310215 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1037fb34-a43a-4a7e-b838-595832697545 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:35.310600 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1037fb34-a43a-4a7e-b838-595832697545 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:35.311507 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1037fb34-a43a-4a7e-b838-595832697545 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:35.311730 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1037fb34-a43a-4a7e-b838-595832697545 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:35.312173 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-1037fb34-a43a-4a7e-b838-595832697545 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:35.440206 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-1037fb34-a43a-4a7e-b838-595832697545 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:35.448130 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-1037fb34-a43a-4a7e-b838-595832697545 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1266 microversion: 2.1 time: 0.149830 Aug 21 01:39:35.448683 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 586/1155] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:35 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:36.097016 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-41828601-868d-4415-9a08-73e9d4f44fe1 req-c6dc3a66-fb51-4e96-b4f5-44a9875b66e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f76f93df-1be7-4260-bcb6-95563c549933", "name": "network-vif-plugged", "status": "completed", "tag": "e5006064-c051-4d6e-b083-787006f20c1f"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:39:36.106569 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-41828601-868d-4415-9a08-73e9d4f44fe1 req-c6dc3a66-fb51-4e96-b4f5-44a9875b66e5 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:36.107101 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-41828601-868d-4415-9a08-73e9d4f44fe1 req-c6dc3a66-fb51-4e96-b4f5-44a9875b66e5 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:36.107481 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-41828601-868d-4415-9a08-73e9d4f44fe1 req-c6dc3a66-fb51-4e96-b4f5-44a9875b66e5 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:36.118386 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-41828601-868d-4415-9a08-73e9d4f44fe1 req-c6dc3a66-fb51-4e96-b4f5-44a9875b66e5 service nova] Creating event network-vif-plugged:e5006064-c051-4d6e-b083-787006f20c1f for instance f76f93df-1be7-4260-bcb6-95563c549933 on hv-cinder-maste Aug 21 01:39:36.123553 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-41828601-868d-4415-9a08-73e9d4f44fe1 req-c6dc3a66-fb51-4e96-b4f5-44a9875b66e5 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030391 Aug 21 01:39:36.124199 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 570/1156] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:39:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:39:36.469807 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-d937d192-4930-41ca-b8e5-0c1ffdc24dca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:36.474008 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-d937d192-4930-41ca-b8e5-0c1ffdc24dca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:36.476971 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d937d192-4930-41ca-b8e5-0c1ffdc24dca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:36.477452 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d937d192-4930-41ca-b8e5-0c1ffdc24dca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:36.477885 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d937d192-4930-41ca-b8e5-0c1ffdc24dca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:36.478565 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d937d192-4930-41ca-b8e5-0c1ffdc24dca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:36.479026 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d937d192-4930-41ca-b8e5-0c1ffdc24dca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:36.479522 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-d937d192-4930-41ca-b8e5-0c1ffdc24dca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:36.673069 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-d937d192-4930-41ca-b8e5-0c1ffdc24dca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:36.681523 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-d937d192-4930-41ca-b8e5-0c1ffdc24dca tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1266 microversion: 2.1 time: 0.216035 Aug 21 01:39:36.682105 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 587/1157] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:36 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1266 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:37.703658 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-d3fbebad-76e3-4709-a1a5-f92abc8b003d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:37.704396 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-d3fbebad-76e3-4709-a1a5-f92abc8b003d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:37.711055 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d3fbebad-76e3-4709-a1a5-f92abc8b003d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:37.711548 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d3fbebad-76e3-4709-a1a5-f92abc8b003d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:37.712654 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d3fbebad-76e3-4709-a1a5-f92abc8b003d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:37.713406 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d3fbebad-76e3-4709-a1a5-f92abc8b003d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:37.713868 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d3fbebad-76e3-4709-a1a5-f92abc8b003d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:37.714294 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-d3fbebad-76e3-4709-a1a5-f92abc8b003d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:37.847599 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-d3fbebad-76e3-4709-a1a5-f92abc8b003d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:37.855232 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-d3fbebad-76e3-4709-a1a5-f92abc8b003d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1266 microversion: 2.1 time: 0.156558 Aug 21 01:39:37.855796 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 571/1158] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:37 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:38.874826 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-f1379037-ddb1-4540-b6f4-0f2a1ca721d2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:38.875432 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-f1379037-ddb1-4540-b6f4-0f2a1ca721d2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:38.882164 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f1379037-ddb1-4540-b6f4-0f2a1ca721d2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:38.882643 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f1379037-ddb1-4540-b6f4-0f2a1ca721d2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:38.883074 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f1379037-ddb1-4540-b6f4-0f2a1ca721d2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:38.884265 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f1379037-ddb1-4540-b6f4-0f2a1ca721d2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:38.884716 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f1379037-ddb1-4540-b6f4-0f2a1ca721d2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:38.885135 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-f1379037-ddb1-4540-b6f4-0f2a1ca721d2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:39.015488 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-f1379037-ddb1-4540-b6f4-0f2a1ca721d2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:39.023183 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-f1379037-ddb1-4540-b6f4-0f2a1ca721d2 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1453 microversion: 2.1 time: 0.152203 Aug 21 01:39:39.023752 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 588/1159] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:38 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1453 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:39.043626 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-06de0156-fd89-4888-ad5e-43a67b91fc3d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:39:39.044311 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-06de0156-fd89-4888-ad5e-43a67b91fc3d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:39:39.050721 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06de0156-fd89-4888-ad5e-43a67b91fc3d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:39.051211 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06de0156-fd89-4888-ad5e-43a67b91fc3d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:39.051669 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06de0156-fd89-4888-ad5e-43a67b91fc3d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:39.052377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06de0156-fd89-4888-ad5e-43a67b91fc3d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:39.052839 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06de0156-fd89-4888-ad5e-43a67b91fc3d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:39.053271 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-06de0156-fd89-4888-ad5e-43a67b91fc3d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:39.184799 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-06de0156-fd89-4888-ad5e-43a67b91fc3d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:39:39.192500 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-06de0156-fd89-4888-ad5e-43a67b91fc3d tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1453 microversion: 2.1 time: 0.152715 Aug 21 01:39:39.193062 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 572/1160] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:39:39 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1453 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:39:41.222021 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-5343aa49-dbe4-4a12-ac5a-66f3f6cd9ea1 req-743e6d49-5677-452d-ad3a-4dd517a1d563 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f76f93df-1be7-4260-bcb6-95563c549933", "tag": "e5006064-c051-4d6e-b083-787006f20c1f"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:39:41.232560 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-5343aa49-dbe4-4a12-ac5a-66f3f6cd9ea1 req-743e6d49-5677-452d-ad3a-4dd517a1d563 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:39:41.233115 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-5343aa49-dbe4-4a12-ac5a-66f3f6cd9ea1 req-743e6d49-5677-452d-ad3a-4dd517a1d563 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:39:41.233610 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-5343aa49-dbe4-4a12-ac5a-66f3f6cd9ea1 req-743e6d49-5677-452d-ad3a-4dd517a1d563 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:39:41.245436 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-5343aa49-dbe4-4a12-ac5a-66f3f6cd9ea1 req-743e6d49-5677-452d-ad3a-4dd517a1d563 service nova] Creating event network-changed:e5006064-c051-4d6e-b083-787006f20c1f for instance f76f93df-1be7-4260-bcb6-95563c549933 on hv-cinder-maste Aug 21 01:39:41.250833 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-5343aa49-dbe4-4a12-ac5a-66f3f6cd9ea1 req-743e6d49-5677-452d-ad3a-4dd517a1d563 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032418 Aug 21 01:39:41.251374 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 589/1161] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:39:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:41:17.733547 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:41:17.734190 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Sun Aug 21 01:41:17 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:41:21.079209 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR oslo.messaging._drivers.impl_rabbit [-] [50ee8623-80f2-4b72-acbd-a0882be55052] AMQP server on 10.222.0.47:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed Aug 21 01:41:21.080134 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_api:SELECT=22 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:41:21.081364 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=3 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:41:21.082356 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG dbcounter [-] [121277] Writing DB stats nova_cell1:SELECT=58 {{(pid=121277) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:41:21.082847 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG dbcounter [-] [121276] Writing DB stats nova_cell1:SELECT=46 {{(pid=121276) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 21 01:41:21.086814 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-1c57d92d-8358-4bd8-ad1b-8025adf74abc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:21.087345 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1c57d92d-8358-4bd8-ad1b-8025adf74abc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:21.094640 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-69e145b7-429f-439f-8b79-787dfc4119f5 req-13ed5cb0-9f2b-40b8-ac44-c9010536202d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f76f93df-1be7-4260-bcb6-95563c549933", "tag": "e5006064-c051-4d6e-b083-787006f20c1f"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:41:21.102799 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c57d92d-8358-4bd8-ad1b-8025adf74abc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:21.103447 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c57d92d-8358-4bd8-ad1b-8025adf74abc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:21.104764 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-69e145b7-429f-439f-8b79-787dfc4119f5 req-13ed5cb0-9f2b-40b8-ac44-c9010536202d service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:21.105248 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-69e145b7-429f-439f-8b79-787dfc4119f5 req-13ed5cb0-9f2b-40b8-ac44-c9010536202d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:21.105671 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-69e145b7-429f-439f-8b79-787dfc4119f5 req-13ed5cb0-9f2b-40b8-ac44-c9010536202d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:21.107263 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c57d92d-8358-4bd8-ad1b-8025adf74abc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:21.108829 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c57d92d-8358-4bd8-ad1b-8025adf74abc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:21.109481 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c57d92d-8358-4bd8-ad1b-8025adf74abc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:21.110123 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-1c57d92d-8358-4bd8-ad1b-8025adf74abc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:21.118301 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-69e145b7-429f-439f-8b79-787dfc4119f5 req-13ed5cb0-9f2b-40b8-ac44-c9010536202d service nova] Creating event network-changed:e5006064-c051-4d6e-b083-787006f20c1f for instance f76f93df-1be7-4260-bcb6-95563c549933 on hv-cinder-maste Aug 21 01:41:21.124741 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-69e145b7-429f-439f-8b79-787dfc4119f5 req-13ed5cb0-9f2b-40b8-ac44-c9010536202d service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046401 Aug 21 01:41:21.125658 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 590/1162] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:41:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:41:21.139122 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-1c57d92d-8358-4bd8-ad1b-8025adf74abc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Going to try to terminate instance {{(pid=121276) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:41:21.541115 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-1c57d92d-8358-4bd8-ad1b-8025adf74abc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "DELETE /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 204 len: 0 microversion: 2.1 time: 0.466279 Aug 21 01:41:21.554837 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8b833e56-4e61-4c3f-8696-c83fdc25b984 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:21.556415 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 573/1163] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:41:21 2022] DELETE /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:41:21.556692 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8b833e56-4e61-4c3f-8696-c83fdc25b984 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:21.569644 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8b833e56-4e61-4c3f-8696-c83fdc25b984 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:21.570408 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8b833e56-4e61-4c3f-8696-c83fdc25b984 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:21.570896 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8b833e56-4e61-4c3f-8696-c83fdc25b984 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:21.571605 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8b833e56-4e61-4c3f-8696-c83fdc25b984 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:21.572196 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8b833e56-4e61-4c3f-8696-c83fdc25b984 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:21.572801 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8b833e56-4e61-4c3f-8696-c83fdc25b984 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:21.780676 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-8b833e56-4e61-4c3f-8696-c83fdc25b984 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:41:21.792634 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8b833e56-4e61-4c3f-8696-c83fdc25b984 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1577 microversion: 2.1 time: 0.244034 Aug 21 01:41:21.793373 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 591/1164] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:41:21 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1577 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:41:22.815131 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-ce6fad90-4356-4537-adae-b6e34d7b0847 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:22.815827 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-ce6fad90-4356-4537-adae-b6e34d7b0847 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:22.822551 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce6fad90-4356-4537-adae-b6e34d7b0847 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:22.823033 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce6fad90-4356-4537-adae-b6e34d7b0847 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:22.823469 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce6fad90-4356-4537-adae-b6e34d7b0847 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:22.824211 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce6fad90-4356-4537-adae-b6e34d7b0847 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:22.824664 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce6fad90-4356-4537-adae-b6e34d7b0847 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:22.825088 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-ce6fad90-4356-4537-adae-b6e34d7b0847 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:23.006437 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-ce6fad90-4356-4537-adae-b6e34d7b0847 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:41:23.014654 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-ce6fad90-4356-4537-adae-b6e34d7b0847 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1577 microversion: 2.1 time: 0.204477 Aug 21 01:41:23.015222 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 574/1165] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:41:22 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1577 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:41:24.035831 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-e1f6d73c-d23f-41cf-a67d-ae3a96617540 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:24.036470 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-e1f6d73c-d23f-41cf-a67d-ae3a96617540 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:24.044471 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e1f6d73c-d23f-41cf-a67d-ae3a96617540 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:24.044957 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e1f6d73c-d23f-41cf-a67d-ae3a96617540 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:24.045874 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e1f6d73c-d23f-41cf-a67d-ae3a96617540 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:24.046571 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e1f6d73c-d23f-41cf-a67d-ae3a96617540 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:24.047020 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e1f6d73c-d23f-41cf-a67d-ae3a96617540 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:24.047440 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-e1f6d73c-d23f-41cf-a67d-ae3a96617540 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:24.064449 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO oslo.messaging._drivers.impl_rabbit [-] [50ee8623-80f2-4b72-acbd-a0882be55052] Reconnected to AMQP server on 10.222.0.47:5672 via [amqp] client with port 33676. Aug 21 01:41:24.295164 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-e1f6d73c-d23f-41cf-a67d-ae3a96617540 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:41:24.303452 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-e1f6d73c-d23f-41cf-a67d-ae3a96617540 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1577 microversion: 2.1 time: 0.271445 Aug 21 01:41:24.304019 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 592/1166] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:41:24 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1577 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:41:24.717105 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-3eec77d3-b73a-4023-909f-f5a9bd98f83b req-af3b00dd-dfcb-4e88-82a1-7f185cae360d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f76f93df-1be7-4260-bcb6-95563c549933", "name": "network-vif-deleted", "tag": "e5006064-c051-4d6e-b083-787006f20c1f"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:41:24.727020 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-3eec77d3-b73a-4023-909f-f5a9bd98f83b req-af3b00dd-dfcb-4e88-82a1-7f185cae360d service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:24.727492 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-3eec77d3-b73a-4023-909f-f5a9bd98f83b req-af3b00dd-dfcb-4e88-82a1-7f185cae360d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:24.727934 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-3eec77d3-b73a-4023-909f-f5a9bd98f83b req-af3b00dd-dfcb-4e88-82a1-7f185cae360d service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:24.738881 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-3eec77d3-b73a-4023-909f-f5a9bd98f83b req-af3b00dd-dfcb-4e88-82a1-7f185cae360d service nova] Creating event network-vif-deleted:e5006064-c051-4d6e-b083-787006f20c1f for instance f76f93df-1be7-4260-bcb6-95563c549933 on hv-cinder-maste Aug 21 01:41:24.743643 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-3eec77d3-b73a-4023-909f-f5a9bd98f83b req-af3b00dd-dfcb-4e88-82a1-7f185cae360d service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082789 Aug 21 01:41:24.744245 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 575/1167] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:41:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:41:25.324578 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-adc659d7-5f57-4e59-8b55-7af92ce36dae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:25.325195 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-adc659d7-5f57-4e59-8b55-7af92ce36dae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:25.332909 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-adc659d7-5f57-4e59-8b55-7af92ce36dae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:25.334866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-adc659d7-5f57-4e59-8b55-7af92ce36dae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:25.334866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-adc659d7-5f57-4e59-8b55-7af92ce36dae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:25.334866 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-adc659d7-5f57-4e59-8b55-7af92ce36dae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:25.335273 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-adc659d7-5f57-4e59-8b55-7af92ce36dae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:25.335380 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-adc659d7-5f57-4e59-8b55-7af92ce36dae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:25.398800 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-adc659d7-5f57-4e59-8b55-7af92ce36dae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:41:25.406644 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-adc659d7-5f57-4e59-8b55-7af92ce36dae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 200 len: 1225 microversion: 2.1 time: 0.085988 Aug 21 01:41:25.407235 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 593/1168] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:41:25 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:41:26.427912 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fca7b382-07c1-4d2d-88f4-0e4dc1fa9cb1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:26.428643 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-fca7b382-07c1-4d2d-88f4-0e4dc1fa9cb1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: f76f93df-1be7-4260-bcb6-95563c549933] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:26.435223 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fca7b382-07c1-4d2d-88f4-0e4dc1fa9cb1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:26.435750 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fca7b382-07c1-4d2d-88f4-0e4dc1fa9cb1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:26.436216 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fca7b382-07c1-4d2d-88f4-0e4dc1fa9cb1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:26.437054 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fca7b382-07c1-4d2d-88f4-0e4dc1fa9cb1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:26.437563 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fca7b382-07c1-4d2d-88f4-0e4dc1fa9cb1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:26.437985 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-fca7b382-07c1-4d2d-88f4-0e4dc1fa9cb1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:26.452307 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-fca7b382-07c1-4d2d-88f4-0e4dc1fa9cb1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] HTTP exception thrown: Instance f76f93df-1be7-4260-bcb6-95563c549933 could not be found. Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fca7b382-07c1-4d2d-88f4-0e4dc1fa9cb1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Request method failure captured: Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 HTTP/1.1 Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYufRkMP3i_c3e0Vr_zgnzF_eSwVsk8M2N0eQOxptmM6V3DchJQy0M67Fr9G61C8FQolpg4xfXTwdedQKOqV9nQf-UOv9V2jyiY57zjWmPWe7uvA2VzMRuLDZd_CPbp7XgOPBjzm_Eiw1Dk-g2GXgEtB0MrihdzJ7mHtU79rwyzdn7A Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 4124766f4f2843229059dbbdf7a2de74 Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:41:26.453950 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:41:26.456268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:41:26.456268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:41:26.456268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 4124766f4f2843229059dbbdf7a2de74 Aug 21 01:41:26.456268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:41:26.456268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-TestVolumeBootPattern-2085187589-project Aug 21 01:41:26.456268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:41:26.456268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:41:26.456268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: fef6461d7ca14cab829412ce3b53852a Aug 21 01:41:26.456268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-TestVolumeBootPattern-2085187589-project Aug 21 01:41:26.456268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:41:26.456268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance f76f93df-1be7-4260-bcb6-95563c549933 could not be found. Aug 21 01:41:26.456268 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': 'f76f93df-1be7-4260-bcb6-95563c549933'} Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f76f93df-1be7-4260-bcb6-95563c549933 could not be found. Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:41:26.457711 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f76f93df-1be7-4260-bcb6-95563c549933 could not be found. Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f76f93df-1be7-4260-bcb6-95563c549933 could not be found. Aug 21 01:41:26.459409 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:41:26.460956 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-fca7b382-07c1-4d2d-88f4-0e4dc1fa9cb1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Returning 404 to user: Instance f76f93df-1be7-4260-bcb6-95563c549933 could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:41:26.460956 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-fca7b382-07c1-4d2d-88f4-0e4dc1fa9cb1 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933" status: 404 len: 111 microversion: 2.1 time: 0.032679 Aug 21 01:41:26.460956 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 576/1169] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:41:26 2022] GET /compute/v2.1/servers/f76f93df-1be7-4260-bcb6-95563c549933 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:41:30.088982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-478bb3f4-1910-4346-ac13-ace0332f53ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:30.090476 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-478bb3f4-1910-4346-ac13-ace0332f53ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:30.090719 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [req-6290d685-8bfd-42ab-be2a-0ea7eb47c0ea req-3e38cc2a-fb80-4380-b3f5-415bf2f53bc5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb", "tag": "6fc060b4-65bc-403a-be6a-d0408f800a0a"}]} {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:41:30.098519 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-478bb3f4-1910-4346-ac13-ace0332f53ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:30.099099 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-478bb3f4-1910-4346-ac13-ace0332f53ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:30.099607 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-478bb3f4-1910-4346-ac13-ace0332f53ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:30.100800 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-478bb3f4-1910-4346-ac13-ace0332f53ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:30.101478 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-478bb3f4-1910-4346-ac13-ace0332f53ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:30.101975 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-478bb3f4-1910-4346-ac13-ace0332f53ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:30.102085 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-6290d685-8bfd-42ab-be2a-0ea7eb47c0ea req-3e38cc2a-fb80-4380-b3f5-415bf2f53bc5 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:30.102533 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-6290d685-8bfd-42ab-be2a-0ea7eb47c0ea req-3e38cc2a-fb80-4380-b3f5-415bf2f53bc5 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:30.103003 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [req-6290d685-8bfd-42ab-be2a-0ea7eb47c0ea req-3e38cc2a-fb80-4380-b3f5-415bf2f53bc5 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:30.114674 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.compute.server_external_events [req-6290d685-8bfd-42ab-be2a-0ea7eb47c0ea req-3e38cc2a-fb80-4380-b3f5-415bf2f53bc5 service nova] Creating event network-changed:6fc060b4-65bc-403a-be6a-d0408f800a0a for instance 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb on hv-cinder-maste Aug 21 01:41:30.120261 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [req-6290d685-8bfd-42ab-be2a-0ea7eb47c0ea req-3e38cc2a-fb80-4380-b3f5-415bf2f53bc5 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033035 Aug 21 01:41:30.121127 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 577/1170] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:41:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:41:30.129254 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-478bb3f4-1910-4346-ac13-ace0332f53ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Going to try to terminate instance {{(pid=121277) delete /opt/stack/nova/nova/compute/api.py:2706}} Aug 21 01:41:30.411512 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-478bb3f4-1910-4346-ac13-ace0332f53ae tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "DELETE /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 204 len: 0 microversion: 2.1 time: 0.325869 Aug 21 01:41:30.412355 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 594/1171] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:41:30 2022] DELETE /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 0 bytes in 328 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 21 01:41:30.422047 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-9c56ec00-23ac-4cf9-a407-1d5c62a4a576 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:30.422919 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-9c56ec00-23ac-4cf9-a407-1d5c62a4a576 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:30.430177 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c56ec00-23ac-4cf9-a407-1d5c62a4a576 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:30.430893 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c56ec00-23ac-4cf9-a407-1d5c62a4a576 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:30.432308 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c56ec00-23ac-4cf9-a407-1d5c62a4a576 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:30.433023 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c56ec00-23ac-4cf9-a407-1d5c62a4a576 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:30.433482 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c56ec00-23ac-4cf9-a407-1d5c62a4a576 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:30.433903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-9c56ec00-23ac-4cf9-a407-1d5c62a4a576 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:30.574937 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-9c56ec00-23ac-4cf9-a407-1d5c62a4a576 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:41:30.583771 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-9c56ec00-23ac-4cf9-a407-1d5c62a4a576 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1576 microversion: 2.1 time: 0.166716 Aug 21 01:41:30.584377 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 578/1172] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:41:30 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1576 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:41:31.605670 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-4f6a53ea-c5c2-4266-a914-162647624fd7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:31.606289 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-4f6a53ea-c5c2-4266-a914-162647624fd7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:31.612976 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f6a53ea-c5c2-4266-a914-162647624fd7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:31.613569 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f6a53ea-c5c2-4266-a914-162647624fd7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:31.614013 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f6a53ea-c5c2-4266-a914-162647624fd7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:31.614714 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f6a53ea-c5c2-4266-a914-162647624fd7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:31.615173 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f6a53ea-c5c2-4266-a914-162647624fd7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:31.615628 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-4f6a53ea-c5c2-4266-a914-162647624fd7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:31.745999 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.policy [None req-4f6a53ea-c5c2-4266-a914-162647624fd7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121277) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:41:31.754375 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-4f6a53ea-c5c2-4266-a914-162647624fd7 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1576 microversion: 2.1 time: 0.153047 Aug 21 01:41:31.754934 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 595/1173] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:41:31 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1576 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:41:32.775531 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-3c2cc104-0367-4cf1-a51a-e54da7220961 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:32.776163 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-3c2cc104-0367-4cf1-a51a-e54da7220961 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:32.782803 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3c2cc104-0367-4cf1-a51a-e54da7220961 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:32.783897 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3c2cc104-0367-4cf1-a51a-e54da7220961 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:32.784691 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3c2cc104-0367-4cf1-a51a-e54da7220961 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:32.785736 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3c2cc104-0367-4cf1-a51a-e54da7220961 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:32.786823 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3c2cc104-0367-4cf1-a51a-e54da7220961 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:32.786823 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-3c2cc104-0367-4cf1-a51a-e54da7220961 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:32.988990 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-3c2cc104-0367-4cf1-a51a-e54da7220961 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:41:32.997139 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-3c2cc104-0367-4cf1-a51a-e54da7220961 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1458 microversion: 2.1 time: 0.225392 Aug 21 01:41:32.997717 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 579/1174] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:41:32 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1458 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 21 01:41:33.000673 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [req-dea4e5d1-4734-44f1-b838-c85506566b45 req-5b60dacc-2425-48fa-97b0-70e3d9e85999 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb", "name": "network-vif-deleted", "tag": "6fc060b4-65bc-403a-be6a-d0408f800a0a"}]} {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 21 01:41:33.010958 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-dea4e5d1-4734-44f1-b838-c85506566b45 req-5b60dacc-2425-48fa-97b0-70e3d9e85999 service nova] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:33.011442 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-dea4e5d1-4734-44f1-b838-c85506566b45 req-5b60dacc-2425-48fa-97b0-70e3d9e85999 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:33.011876 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [req-dea4e5d1-4734-44f1-b838-c85506566b45 req-5b60dacc-2425-48fa-97b0-70e3d9e85999 service nova] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:33.024859 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.compute.server_external_events [req-dea4e5d1-4734-44f1-b838-c85506566b45 req-5b60dacc-2425-48fa-97b0-70e3d9e85999 service nova] Creating event network-vif-deleted:6fc060b4-65bc-403a-be6a-d0408f800a0a for instance 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb on hv-cinder-maste Aug 21 01:41:33.030371 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [req-dea4e5d1-4734-44f1-b838-c85506566b45 req-5b60dacc-2425-48fa-97b0-70e3d9e85999 service nova] 10.222.0.47 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033462 Aug 21 01:41:33.031349 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 596/1175] 10.222.0.47 () {64 vars in 1317 bytes} [Sun Aug 21 01:41:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 21 01:41:34.018234 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:34.018835 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:34.025322 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:34.025803 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:34.026240 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:34.026934 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:34.027396 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:34.027817 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:34.062317 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:34.062792 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:34.063216 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:34.063718 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.objects.instance [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lazy-loading 'fault' on Instance uuid 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb {{(pid=121276) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 21 01:41:34.104340 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.policy [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fef6461d7ca14cab829412ce3b53852a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4124766f4f2843229059dbbdf7a2de74', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121276) authorize /opt/stack/nova/nova/policy.py:199}} Aug 21 01:41:34.112045 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-f4c67600-f7b1-482d-bb44-82290d541a29 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 200 len: 1230 microversion: 2.1 time: 0.097805 Aug 21 01:41:34.113402 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 580/1176] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:41:34 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 1230 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 21 01:41:35.135345 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-51ad59c9-6978-440f-a3d0-29e2504f469e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:35.135965 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-51ad59c9-6978-440f-a3d0-29e2504f469e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:35.142140 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51ad59c9-6978-440f-a3d0-29e2504f469e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:35.142793 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51ad59c9-6978-440f-a3d0-29e2504f469e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:35.143240 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51ad59c9-6978-440f-a3d0-29e2504f469e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:35.143929 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51ad59c9-6978-440f-a3d0-29e2504f469e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:35.144409 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51ad59c9-6978-440f-a3d0-29e2504f469e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:35.144834 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-51ad59c9-6978-440f-a3d0-29e2504f469e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:35.158627 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-51ad59c9-6978-440f-a3d0-29e2504f469e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] HTTP exception thrown: Instance 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb could not be found. Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-51ad59c9-6978-440f-a3d0-29e2504f469e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Request method failure captured: Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb HTTP/1.1 Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYufRkMP3i_c3e0Vr_zgnzF_eSwVsk8M2N0eQOxptmM6V3DchJQy0M67Fr9G61C8FQolpg4xfXTwdedQKOqV9nQf-UOv9V2jyiY57zjWmPWe7uvA2VzMRuLDZd_CPbp7XgOPBjzm_Eiw1Dk-g2GXgEtB0MrihdzJ7mHtU79rwyzdn7A Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 4124766f4f2843229059dbbdf7a2de74 Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:41:35.160293 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:41:35.162024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:41:35.162024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:41:35.162024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 4124766f4f2843229059dbbdf7a2de74 Aug 21 01:41:35.162024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:41:35.162024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-TestVolumeBootPattern-2085187589-project Aug 21 01:41:35.162024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:41:35.162024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:41:35.162024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: fef6461d7ca14cab829412ce3b53852a Aug 21 01:41:35.162024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-TestVolumeBootPattern-2085187589-project Aug 21 01:41:35.162024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:41:35.162024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb could not be found. Aug 21 01:41:35.162024 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb'} Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb could not be found. Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:41:35.162982 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb could not be found. Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb could not be found. Aug 21 01:41:35.164655 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.166286 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-51ad59c9-6978-440f-a3d0-29e2504f469e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Returning 404 to user: Instance 13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:41:35.166286 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-51ad59c9-6978-440f-a3d0-29e2504f469e tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb" status: 404 len: 111 microversion: 2.1 time: 0.031213 Aug 21 01:41:35.166286 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 597/1177] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:41:35 2022] GET /compute/v2.1/servers/13cfd4e5-2bd6-4c36-8c9e-561fe8ac31bb => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:41:35.665187 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-afb189fd-5770-44bf-8aa7-addd4941affc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:35.665701 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.compute.api [None req-afb189fd-5770-44bf-8aa7-addd4941affc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121276) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:35.671779 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-afb189fd-5770-44bf-8aa7-addd4941affc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:35.672299 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-afb189fd-5770-44bf-8aa7-addd4941affc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:35.672732 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-afb189fd-5770-44bf-8aa7-addd4941affc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:35.673423 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-afb189fd-5770-44bf-8aa7-addd4941affc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:35.673870 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-afb189fd-5770-44bf-8aa7-addd4941affc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:35.674294 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG oslo_concurrency.lockutils [None req-afb189fd-5770-44bf-8aa7-addd4941affc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121276) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:35.688264 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.wsgi [None req-afb189fd-5770-44bf-8aa7-addd4941affc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] HTTP exception thrown: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-afb189fd-5770-44bf-8aa7-addd4941affc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Request method failure captured: Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: request: DELETE /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be HTTP/1.1 Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept: application/json Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Accept-Encoding: identity Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Connection: close Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Length: 0 Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Content-Type: application/json Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Host: 10.222.0.47 Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Openstack-System-Scope: None Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: User-Agent: python-urllib3/1.26.11 Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Auth-Token: gAAAAABjAYufRkMP3i_c3e0Vr_zgnzF_eSwVsk8M2N0eQOxptmM6V3DchJQy0M67Fr9G61C8FQolpg4xfXTwdedQKOqV9nQf-UOv9V2jyiY57zjWmPWe7uvA2VzMRuLDZd_CPbp7XgOPBjzm_Eiw1Dk-g2GXgEtB0MrihdzJ7mHtU79rwyzdn7A Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Id: None Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Domain-Name: None Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Identity-Status: Confirmed Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Is-Admin-Project: True Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Id: default Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Domain-Name: Default Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Id: 4124766f4f2843229059dbbdf7a2de74 Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Project-Name: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Role: member,reader Aug 21 01:41:35.689903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Roles: member,reader Aug 21 01:41:35.691696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:41:35.691696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:41:35.691696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Id: 4124766f4f2843229059dbbdf7a2de74 Aug 21 01:41:35.691696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-Tenant-Name: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:41:35.691696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User: tempest-TestVolumeBootPattern-2085187589-project Aug 21 01:41:35.691696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Id: default Aug 21 01:41:35.691696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Domain-Name: Default Aug 21 01:41:35.691696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Id: fef6461d7ca14cab829412ce3b53852a Aug 21 01:41:35.691696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: X-User-Name: tempest-TestVolumeBootPattern-2085187589-project Aug 21 01:41:35.691696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: method: > Aug 21 01:41:35.691696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: exception: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:41:35.691696 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: action_args: {'id': '85c51937-c541-455e-a0db-4d06041c67be'} Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:41:35.692903 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:41:35.694497 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.696304 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-afb189fd-5770-44bf-8aa7-addd4941affc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Returning 404 to user: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. {{(pid=121276) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:41:35.696304 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-afb189fd-5770-44bf-8aa7-addd4941affc tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "DELETE /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 404 len: 111 microversion: 2.1 time: 0.030020 Aug 21 01:41:35.696304 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 581/1178] 10.222.0.47 () {58 vars in 1239 bytes} [Sun Aug 21 01:41:35 2022] DELETE /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:41:35.699830 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8fc8b9be-1e6e-4830-91c6-5947d3041939 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '>' {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:35.700493 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.compute.api [None req-8fc8b9be-1e6e-4830-91c6-5947d3041939 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] [instance: 85c51937-c541-455e-a0db-4d06041c67be] Fetching instance by UUID {{(pid=121277) get /opt/stack/nova/nova/compute/api.py:2962}} Aug 21 01:41:35.707322 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8fc8b9be-1e6e-4830-91c6-5947d3041939 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:35.708027 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8fc8b9be-1e6e-4830-91c6-5947d3041939 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:35.708288 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8fc8b9be-1e6e-4830-91c6-5947d3041939 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:35.708992 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8fc8b9be-1e6e-4830-91c6-5947d3041939 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Acquiring lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 21 01:41:35.710261 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8fc8b9be-1e6e-4830-91c6-5947d3041939 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 21 01:41:35.710261 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG oslo_concurrency.lockutils [None req-8fc8b9be-1e6e-4830-91c6-5947d3041939 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Lock "6547be85-e237-4d35-aaa7-703fe51c4e0b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121277) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 21 01:41:35.724282 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.wsgi [None req-8fc8b9be-1e6e-4830-91c6-5947d3041939 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] HTTP exception thrown: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8fc8b9be-1e6e-4830-91c6-5947d3041939 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Request method failure captured: Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: request: GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be HTTP/1.1 Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept: application/json Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Accept-Encoding: identity Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Connection: close Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Length: 0 Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Content-Type: application/json Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Host: 10.222.0.47 Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Openstack-System-Scope: None Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: User-Agent: python-urllib3/1.26.11 Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Auth-Token: gAAAAABjAYufRkMP3i_c3e0Vr_zgnzF_eSwVsk8M2N0eQOxptmM6V3DchJQy0M67Fr9G61C8FQolpg4xfXTwdedQKOqV9nQf-UOv9V2jyiY57zjWmPWe7uvA2VzMRuLDZd_CPbp7XgOPBjzm_Eiw1Dk-g2GXgEtB0MrihdzJ7mHtU79rwyzdn7A Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Id: None Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Domain-Name: None Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Identity-Status: Confirmed Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Is-Admin-Project: True Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Id: default Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Domain-Name: Default Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Id: 4124766f4f2843229059dbbdf7a2de74 Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Project-Name: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Role: member,reader Aug 21 01:41:35.725944 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Roles: member,reader Aug 21 01:41:35.727689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.47:8080", "publicURL": "http://10.222.0.47:8080/v1/AUTH_4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/volume/v3/4124766f4f2843229059dbbdf7a2de74"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.47/identity"}]}] Aug 21 01:41:35.727689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:41:35.727689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Id: 4124766f4f2843229059dbbdf7a2de74 Aug 21 01:41:35.727689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-Tenant-Name: tempest-TestVolumeBootPattern-2085187589 Aug 21 01:41:35.727689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User: tempest-TestVolumeBootPattern-2085187589-project Aug 21 01:41:35.727689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Id: default Aug 21 01:41:35.727689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Domain-Name: Default Aug 21 01:41:35.727689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Id: fef6461d7ca14cab829412ce3b53852a Aug 21 01:41:35.727689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: X-User-Name: tempest-TestVolumeBootPattern-2085187589-project Aug 21 01:41:35.727689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: method: > Aug 21 01:41:35.727689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: exception: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:41:35.727689 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: action_args: {'id': '85c51937-c541-455e-a0db-4d06041c67be'} Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: {{(pid=121277) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2965, in get Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2912, in _get_instance Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2869, in _get_instance_from_cell Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 21 01:41:35.728776 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. Aug 21 01:41:35.730162 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: ERROR nova.api.openstack.wsgi Aug 21 01:41:35.731499 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: DEBUG nova.api.openstack.wsgi [None req-8fc8b9be-1e6e-4830-91c6-5947d3041939 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Returning 404 to user: Instance 85c51937-c541-455e-a0db-4d06041c67be could not be found. {{(pid=121277) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 21 01:41:35.731499 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO nova.api.openstack.requestlog [None req-8fc8b9be-1e6e-4830-91c6-5947d3041939 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be" status: 404 len: 111 microversion: 2.1 time: 0.031325 Aug 21 01:41:35.731499 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: [pid: 121277|app: 0|req: 598/1179] 10.222.0.47 () {58 vars in 1236 bytes} [Sun Aug 21 01:41:35 2022] GET /compute/v2.1/servers/85c51937-c541-455e-a0db-4d06041c67be => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 21 01:41:37.225709 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: DEBUG nova.api.openstack.wsgi [None req-a1f65fb0-0d5a-4c28-840f-537aa68e4a32 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] Calling method '.version_select at 0x7f9da3510940>' {{(pid=121276) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 21 01:41:37.324488 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO nova.api.openstack.requestlog [None req-a1f65fb0-0d5a-4c28-840f-537aa68e4a32 tempest-TestVolumeBootPattern-2085187589 tempest-TestVolumeBootPattern-2085187589-project] 10.222.0.47 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1465137419" status: 202 len: 0 microversion: 2.1 time: 0.102916 Aug 21 01:41:37.325125 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: [pid: 121276|app: 0|req: 582/1180] 10.222.0.47 () {58 vars in 1263 bytes} [Sun Aug 21 01:41:37 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1465137419 => generated 0 bytes in 104 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 21 01:41:55.974794 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:41:55.975397 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Sun Aug 21 01:41:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:41:56.209201 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:41:56.209461 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Sun Aug 21 01:41:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:43:33.839305 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:43:33.839921 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Sun Aug 21 01:43:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:43:53.279152 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:43:53.279846 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Sun Aug 21 01:43:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:44:37.741872 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:44:37.742472 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Sun Aug 21 01:44:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:45:15.982891 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:45:15.983446 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Sun Aug 21 01:45:15 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:45:16.217659 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:45:16.217888 dv-cinder-master-3-iscsi devstack@n-api.service[121277]: Sun Aug 21 01:45:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Aug 21 01:46:53.847993 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 21 01:46:53.848914 dv-cinder-master-3-iscsi devstack@n-api.service[121276]: Sun Aug 21 01:46:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!